Real-Time Voice-to-Voice AI Assistant with Emotion Recognition
Table of contents
šļø Advanced Voice-to-Voice AI Assistant
š Real-Time | Emotion-Aware | Future-Ready
š§ What is it?
An intelligent, real-time Voice-to-Voice AI Assistant that understands your speech, detects your emotions, and responds naturally using dynamic voices and emotional tones ā like a human.
Inspired by futuristic AI models, this assistant combines:
- š§ Real-time speech recognition (Vosk)
- š¤ Emotion-aware text-to-speech (Edge TTS)
- š§ GPT-style natural language response (via Mistral API)
- šµ Optional background music & typing sound
- š¤ Modular, fully threaded, and scalable architecture
āļø Tech Stack
- Python (Core)
- Vosk (STT)
- Edge TTS (TTS)
- Mistral API (LLM)
- Pygame (Audio Engine)
emotion.py(Custom emotional context engine)- Threading, Logging, and Real-time pipeline
š GitHub Repo
š https://github.com/Asnanp/voice-to-voice-ai
š” Features
šÆ Real-time audio transcription and response
š£ļø Emotion detection and emotional voice synthesis
š Continuous conversation loop
š Background typing sound and music
š Low latency and multi-threaded performance
š Use Cases
Virtual AI Companions
Emotion-aware customer service bots
Mental health assistants
Futuristic voice agents for smart homes
š ļø Future Work
š„ Lip-sync to TTS voice
š§¬ Voice cloning support
š Web UI with full frontend-backend integration
š¤ Agent memory and personalization
š Credits
Built by Asnan P.
16 y/o Self-taught AI Engineer from Kerala, India
GitHub: https://github.com/asnanp/| LinkedIn: https:/linkedin.com/in/asnanp1