Bobcat: Your AI Companion That Sings Your Story

Bobcat is not just a gadget—it's your personal, empathetic AI friend that listens, responds, and creates songs tailored to your emotions and personal story. Conceived during Duke AI Hackathon 2024, this innovative project merges art, technology, and human connection.

Abstract

Bobcat is a groundbreaking AI companion that seamlessly integrates advanced technology with human-centered design to create an emotionally intelligent and artistic user experience. Developed during the Duke AI Hackathon 2024, Bobcat is a portable device that listens to users' thoughts and feelings through voice input and facial recognition, analyzes their sentiments in real-time, and responds with personalized conversations and songs. By blending real-time mood detection with AI-generated music and lyrics, Bobcat redefines the role of AI from a tool to an empathetic companion. Its charm lies in its ability to bridge the gap between technology and emotional connection, fostering creativity, comfort, and joy. Equipped with cutting-edge technologies like DeepFace for facial recognition, sentiment analysis, and UDIO for personalized song creation, Bobcat is designed to resonate emotionally and artistically with users. With potential applications spanning mental health support, creative expression, and digital scalability, Bobcat stands as a testament to the transformative power of AI when merged with art and design.

Presentation & WriteUp

You can find our project Devpost Page here: Devpost

You can find our project Source code here: Github

Value Proposition

Bobcat isn’t just another gadget; it’s a heartfelt, artistic companion that fulfills the fundamental human need for connection and creative expression. In a world where technology is often practical but impersonal, Bobcat shifts that narrative. It’s not a sterile tool but an engaging and empathetic muse that embodies the intersection of art, media, and design. With Bobcat, AI becomes more than an assistant — it transforms into a partner that brings joy, creativity, and comfort through personalized, artistic interaction. It’s a friend to share your day with, who responds with emotional insight and creative flair.

Why is this important? Because in today’s society, where stress and isolation are common, people crave technology that resonates emotionally and creatively. We’re living in an era where the fusion of art, technology, and mental well-being is more relevant than ever. Bobcat brings a touch of artistry to AI, offering a dynamic and media-rich experience that soothes the soul, uplifts the spirit, and fills the gap left by conventional tech products. It’s designed not just to assist, but to inspire and foster genuine connection through the universal language of art and music.

With Bobcat, you’re not just using technology—you’re collaborating with it to make art that feels alive.

Technical Accomplishments

Now, let’s talk about how we brought Bobcat to life. Its intelligence lies in its seamless integration of advanced software and hardware, all within a charming, 3D-printed shell. Here’s what powers Bobcat:

1. Real-Time Facial Recognition: Using DeepFace, Bobcat reads your facial expressions to understand your mood.
2. Voice Transcription: With Speech Recognition, it transcribes your spoken words into text, ensuring it understands your thoughts.
3. Sentiment Analysis: By analyzing both your words and your expressions, Bobcat gauges your mood to tailor its response.
4. Empathetic Chat Responses: Powered by OpenAI 3.5 Turbo and Text-to-Speech (TTS-1-HD), Bobcat creates conversations that feel human, natural, and comforting.
5. Song and Lyrics Generation: Here’s where the magic happens — UDIO composes a song with lyrics and melody that perfectly fit your current state of mind, bringing the song to life with a beautiful, warm voice that resonates with your emotions.

Under the hood, Bobcat runs on a Raspberry Pi 4B, giving it both brains and brawn. Equipped with an integrated microphone and an Arducam IMX708 camera, it’s built to listen, see, and respond with empathy. Audio comes to life through an Audio Core HAT WM8060, delivering high-quality sound that feels as warm as a real conversation. And with a lithium battery module, Bobcat stays powered on the go, ready to be your creative companion anywhere, anytime. This complete integration was no small feat; it took intricate work to achieve real-time operations smoothly, even within the constraints of a 48-hour hackathon.

Project Pipeline

Bobcat’s real-time operations rely on seamless integration:

1. Facial and Voice Input ➡️ Mood Detection (DeepFace + Sentiment Analysis)
2. Transcription & Interpretation ➡️ Chat Response (Speech Recognition + GPT-3.5 Turbo + TTS)
3. Song Composition ➡️ Performance Output (UDIO)

Key Features

• Multimodal Sentiment Analysis: Combines DeepFace for facial recognition and speech-based sentiment analysis for a comprehensive understanding of your mood.
• Voice Transcription: Translates spoken input into text using Speech Recognition, enabling contextual awareness.
• Empathetic AI Interaction: Chat responses powered by OpenAI GPT-3.5 Turbo and Text-to-Speech (TTS-1-HD) mimic natural, human-like conversation.
• Dynamic Song Generation: Utilizes UDIO to compose and sing personalized songs that align with your emotional state, creating a unique, mood-reflective experience.
• Portable & Engaging Design: Encased in a 3D-printed body with a friendly aesthetic that invites interaction.

Hardware

• Raspberry Pi 4B: The processing hub that powers Bobcat.
• Arducam IMX708 Camera: Captures facial expressions for mood detection.
• Audio Core HAT WM8060: Delivers high-quality sound for an immersive experience.
• Integrated Microphone and Speakers: Enable voice interaction and audio feedback.
• Lithium Battery Module: Ensures portability and long-lasting operation.

Software Stack

• DeepFace: Facial recognition for real-time mood analysis.
• Speech Recognition Library: Transcribes voice input to text.
• OpenAI GPT-3.5 Turbo: Empathetic, AI-driven conversation.
• Text-to-Speech (TTS-1-HD): Natural, expressive vocal synthesis.
• UDIO: Generates and performs songs tailored to the detected emotional state.

Novelty

What makes Bobcat truly stand out? Bobcat isn’t just a chatbot or a music player. It’s an entirely new category of product — a real-time, AI-powered friend that listens, feels, and creates. Imagine interacting with a companion that doesn’t just respond, but sings a personalized song tailored to your mood. It’s like having an AI artist who knows you intimately, combining real-time mood detection with music generation in a way that hasn’t been done before.

Bobcat’s charm lies in its combination of:
● Real-time, multimodal sentiment analysis (facial expression and voice input).
● Unique, user-tailored responses that go beyond simple conversation to actual musical creation.
● Physical form factor that’s cute and friendly, making it feel like a living toy you can bond with.

Competitive Advantage

While voice assistants like Alexa and Siri excel in practical functions such as controlling smart devices or answering queries, they lack the emotional intelligence and creativity that Bobcat brings. What truly sets Bobcat apart is its physical design—crafted to feel approachable, friendly, and trustworthy. With its cute, charming appearance and soft, responsive interactions, Bobcat invites users to treat it as a companion, not just a tool. Whether you're stressed and in need of comfort or feeling joyful and eager to share your happiness, Bobcat is there to listen, understand, and create a personalized song that reflects your mood. Its physical form enhances this bond, making it feel more like a trusted friend you can turn to in any emotional state. Bobcat’s integration of real-time facial recognition, sentiment analysis and music generation offers a more human-like, emotionally resonant experience that goes beyond simple conversation. This unique fusion of empathy, creativity, and design elevates Bobcat far beyond conventional voice assistants, offering users a companion that not only responds but genuinely connects with their emotions.

Constraints, Future Work, and Potential

We see Bobcat being loved by children who want a playful friend, adults who need a moment of comfort, and seniors who appreciate companionship. But we’re not stopping there. To enhance its reach and capabilities, future work will focus on expanding Bobcat into the digital space through cloud-based functionalities using AWS Lex and Polly. This will allow Bobcat to offer scalable, app-integrated interactions, making it accessible across devices and platforms like Slack and Discord. By moving to the cloud, users will be able to share their personalized Bobcat-generated songs with friends, fostering even richer emotional interactions and making it a companion that resonates on a broader scale. This way, Bobcat can reach anyone, anytime, offering the same heartwarming experience and creating moments of joy and connection that ripple across social media.

Conclusion

To sum up, Bobcat is a unique blend of smart technology, creative design, and heartfelt purpose. It’s not just another gadget; it’s your artistic companion that brings joy, comfort, and a touch of magic to your day. Whether it’s making you smile with a personalized song or responding with genuine empathy, Bobcat turns technology into something deeply human. It bridges the gap between art and connection, proving that AI can be both innovative and warm. Thank you, and step into a new era where tech isn’t just useful, but meaningful — welcome to Bobcat.