Oct 23, 2024●10 reads

Earlent 2.0: AI-Driven Mental Health Companion

d
@dhrumilpatel21612

Inspiration

In today’s fast-paced and high-pressure world, the need for accessible and effective mental health resources has never been more urgent. Modern lifestyles are often accompanied by relentless personal and professional demands, leading to increased stress, anxiety, and other mental health challenges. Constant connectivity, overwhelming information, and societal expectations make it difficult for individuals to effectively disconnect and manage their mental well-being.

With these stressors affecting daily life, we were inspired to create a solution: a software application aimed at addressing these issues. Earlent is an AI-driven mental health companion designed to provide instant, empathetic support to those in need.

Earlent utilizes state-of-the-art cloud infrastructure and advanced AI technologies to develop an empathetic and intelligent virtual companion aimed at enhancing the mental well-being of users. By facilitating compassionate and insightful interactions, the platform seeks to provide immediate and personalized mental health support that is readily accessible from any location. This ensures that individuals in need can easily access the help they require.

What it does

The Earlent application provides eight core features designed to enhance mental health management and promote well-being:

1. Chat Interface:
The heart of the platform is an intuitive, AI-driven chat interface that facilitates empathetic conversations. The AI responds to users' mental health-related inquiries in a compassionate, understanding manner, offering guidance, suggestions, and emotional support. Whether someone feels overwhelmed or needs someone to "talk" to, the chat interface is designed to feel like a personal companion. The chat can also recommend therapists on request. Another feature of the chat interface is its ability to recommend websites likes when asked and not only does it do that, but it presents it in a user-friendly interface design.
Key Features:

Speech-to-Text and Text-to-Speech: Users can interact using voice input and receive audio responses, enhancing accessibility and providing a more natural conversation experience.
Multi-Source Web Search Integration: The AI utilizes Tavily, Bing, Google, and YouTube searches to recommend therapists, articles, videos, and other resources upon request, presented in a user-friendly interface.
Personalized Recommendations: It offers tailored suggestions for well-being routines, mindfulness practices, and stress-relief techniques based on user input and mood.
Location-Based Assistance: The AI can help users find nearby mental health professionals or wellness centers when asked.

Use Case Diagram:

Use Case Diagram

2. Well-being Routines:
Earlent encourages users to maintain daily or weekly routines focused on mental and emotional health. These routines might include mindfulness practices, meditation activities, or stress relief techniques, tailored to individual needs and preferences. This feature makes use of YouTube and Google API to pull resources and display them based on the users' inputted search data

3. Mood Logger:
Users can record and store their daily moods, serving as a digital journal to help track emotional states over time. When a chat session is finalized, the AI analyzes the user's mood based on the conversation and provides immediate insights. This helps users reflect on their emotions and better understand their mental health. While the app offers mood analysis at the end of chats, future updates may include more extensive AI-driven analysis and trend identification based on users' logged data.

4. Schedule Check-in:

Earlent enables users to schedule periodic check-ins to reflect on their mental state and stay on track with managing their mental health. After scheduling a check-in, users will receive timely notifications to remind them of their upcoming sessions. Specifically, the app sends notifications:

1 month before the scheduled check-in
1 day before the scheduled check-in
1 hour before the scheduled check-in

5. Anonymous Login:

Earlent offers an anonymous login option for users who prefer not to create an account or disclose personal information. This feature allows users to access the chat interface and other functionalities without necessarily setting up an account in the application, ensuring privacy and encouraging more individuals to seek support without hesitation. Anonymous users can benefit from the AI-driven chat interface and receive guidance while maintaining complete anonymity.

6. Forgot Password Feature:

Earlent provides a secure "Forgot Password" feature. Users who have forgotten their passwords can enter their registered email addresses to receive a tokenized link. This link allows them to create a new password securely, ensuring they regain access to their accounts without compromising security.

7. Data Management:

Users have control over their data with the ability to download or delete all or selected portions of their chat history with the AI. This allows users to manage their personal information according to their preferences.

8. Account Deletion:

For users who wish to discontinue using the service, Earlent offers an option to delete their accounts completely. This process ensures that all personal data, including chat histories and user profiles, are permanently removed from the system, respecting user privacy and data protection preferences.

In Summary:

Earlent - The Mental Health Companion is an innovative and AI-powered platform designed to provide personalized and empathetic mental health support. By offering features like a responsive chat interface, customizable well-being routines, mood logging with AI analysis, scheduled check-ins with reminders, anonymous login options, secure password recovery, comprehensive data management, and the option to delete accounts, Earlent empowers users to take control of their mental health in a convenient, supportive, and accessible way.

##How we built it

Database: Azure Cosmos DB (v-core) for its robust, scalable database services to manage dynamic data requirements efficiently.
LLM: The Azure OpenAI platform was integrated for generating empathetic, context-aware responses through advanced AI models like GPT 4o mini.
AI Services: Azure Speech Services enable users to interact with the platform using their voice, improving accessibility and user experience. Speech-to-text allowed users to input requests through speech, while text-to-speech provided responses audibly using window.speechSynthesis.
Frontend: React was chosen for its efficiency in building interactive user interfaces, with Vite used to optimize the development experience and MUI (Material-UI) to design a modern, user-friendly interface.
Backend: Flask was chosen to manage backend operations, including API routing and middleware functionalities, due to its lightweight and unopinionated structure.
Agentic Framework: LangChain was integrated to enhance the AI's functionality, enabling the AI to keep track of conversation history and use tools for web search, vector search and map services.
APIs and Services: We utilized several external APIs to enrich the AI's capabilities:
- Google Web Search API: For retrieving relevant search results from Google.
- Bing Search API: To fetch search results from Bing, providing diverse information sources.
- YouTube Search API: For accessing video content relevant to the user's queries.
- Google Places API: To help users find nearby mental health professionals or wellness centers.
- Tavily API: To access specialized or niche information that may not be readily available through mainstream search engines.
AI Vector Search with FAISS: We integrated FAISS (Facebook AI Similarity Search) for efficient similarity search and clustering of dense vectors, enhancing the AI's ability to retrieve relevant information quickly.
Services & Libraries: pywebpush allowed for real-time communication with users through web push notifications, facilitated by user subscriptions managed through the Subscription model. Flask-Mail was used to manage email communications within the app.
CI/CD Docker was used to containerize the application, ensuring consistency across different computing environments. Github Actions was used to optimize the deployment process by wiring it directly to Render's web services.

Challenges we ran into

Throughout the development of the Earlent application, we encountered several challenges that tested our problem-solving skills and adaptability.

1. Azure Resource Management

Setting up, allocating, and deploying Azure resources was more time-consuming than we initially anticipated. Navigating quotas and limits sometimes hindered our progress and required us to adjust our resource utilization strategies. It was indeed a great learning experience.

2. Learning the Flask Framework

As first-time users of the Flask framework, we needed time to familiarize ourselves with its patterns and structures. The initial learning curve was steep, but the comprehensive documentation and intuitive syntax facilitated our understanding. This experience broadened our expertise in backend development and will undoubtedly benefit future projects.

3. Implementing LangChain Agents

One of the most significant challenges was building an AI agent using LangChain that effectively interacts with users through various tools. LangChain's innovative architecture allows AI models to function as "reasoning machines," leveraging tools like web search APIs and vector databases to generate more informed responses. However, to the inconsistencies in the LangChain documentation especially the aspects of details, quality, and versioning, we had to carry out extensive research and trial-and-error. Overcoming these hurdles deepened our comprehension of advanced AI architectures and improved our problem-solving abilities.

4. Integrating Web Push API and Speech Services

Working with services involving the Web Push API and speech-to-text functionality was a new experience for our team. We successfully incorporated real-time communication through web push notifications and enhanced accessibility with speech-to-text and text-to-speech capabilities, significantly improving the user experience.

Accomplishments That We're Proud Of

Throughout the development of Earlent, we achieved several milestones that we're particularly proud of:

1. Advanced AI Integration

Empathetic AI Conversations: Successfully implemented the Azure OpenAI GPT-4o-mini model to provide compassionate, context-aware responses, creating an AI that feels like a personal companion.
Agentic Framework with LangChain: Integrated LangChain to enhance the AI's reasoning capabilities, enabling it to utilize multiple tools such as web searches, vector searches, and map services within conversations.

2. Innovative Chat Interface Development

Speech-to-Text and Text-to-Speech Features: Developed an intuitive chat interface that supports voice input and audio responses, enhancing accessibility and providing a more natural user experience.
User-Friendly Design: Leveraged React and MUI to create a modern, responsive interface that is both aesthetically pleasing and easy to navigate.

3. Comprehensive API Utilization

Multi-Source Web Search Integration: Incorporated Google Web Search API, Bing Search API, YouTube Search API, Google Places API, and Tavily API to enrich the AI's responses with diverse and relevant information.
Efficient Vector Search with FAISS: Utilized FAISS for fast similarity search and clustering of dense vectors, improving the AI's ability to retrieve pertinent information swiftly.

4. Overcoming Technical Challenges

Azure Resource Management: Navigated the complexities of setting up and deploying Azure resources, overcoming quota limitations, and ensuring scalable infrastructure.
Mastering New Technologies: Gained proficiency in Flask, LangChain, pywebpush, MaterialUI, and Azure Speech Services, despite initial unfamiliarity, to build a robust backend and enhance user experience.
Agent Building with LangChain: Despite documentation inconsistencies we were able to successfully build an AI agent capable of using tools to interact effectively with users.

5. User Privacy and Data Control

Anonymous Login Feature: Implemented an anonymous login option, encouraging users who prefer privacy to access mental health support without providing personal information.
Data Management Capabilities: Enabled users to download or delete all or portions of their chat history, giving them full control over their personal data.
Account Deletion Process: Provided a straightforward way for users to delete their accounts completely, ensuring their data is permanently removed from the system.

6. Real-Time Notifications and Scheduling

Scheduled Check-Ins with Reminders: Developed a scheduling feature that sends users reminders 1 month, 1 day, and 1 hour before their check-ins using pywebpush, promoting consistent engagement and self-care.
Integration of Push Notifications: Successfully implemented real-time communication through web push notifications, enhancing user engagement and interaction with the app.

7. Holistic Mental Health Support Platform

Comprehensive Feature Set: Combined multiple functionalities—empathetic chat interface, mood logging with AI analysis, well-being routines, and scheduled check-ins—to provide a well-rounded mental health support system.
Personalized User Experience: Leveraged AI analysis to offer personalized recommendations and insights, tailoring the app's support to each user's unique needs.

What We Learned

The development journey of Earlent was a transformative experience that enriched our team in numerous ways:

1. Deepening Our Understanding of Advanced AI Technologies

Integrating Large Language Models: Working with Azure OpenAI's GPT-4o-mini allowed us to explore the capabilities of advanced AI models. We learned how to fine-tune the AI to provide empathetic, context-aware responses, which was crucial for creating a supportive mental health companion.
Leveraging LangChain for Agentic AI: Implementing LangChain taught us how to build AI agents capable of reasoning and utilizing tools like web search APIs and vector databases. We navigated its innovative architecture and overcame documentation inconsistencies, enhancing our problem-solving skills.

2. Enhancing User Experience and Accessibility

Implementing Speech-to-Text and Text-to-Speech: We learned how to integrate Azure Speech Services to provide voice interaction capabilities. This not only improved accessibility but also taught us about processing audio data and handling real-time communication.
Designing User-Friendly Interfaces: Utilizing React and MUI, we focused on creating an intuitive and aesthetically pleasing user interface. This experience honed our skills in responsive design and user-centered development.

3. Overcoming Challenges and Adapting to New Technologies

Navigating Documentation Limitations: Facing inconsistencies in LangChain's documentation required us to be resourceful, relying on community forums, experimenting, and iterative testing to achieve our goals.
Learning Under Pressure: Working with new technologies like pywebpush and Azure Speech Services pushed us out of our comfort zones. We embraced the steep learning curves, which ultimately broadened our technical expertise.

4. Effective Collaboration and Team Growth

Distributed Teamwork: Coordinating tasks and integrating different components of the project taught us the value of clear communication and collaboration tools.
Problem-Solving as a Team: Tackling complex challenges together strengthened our teamwork and highlighted the importance of diverse perspectives in finding innovative solutions.

5. Emphasizing User-Centered Design

Understanding User Needs: Developing features like mood analysis and personalized recommendations underscored the importance of empathizing with users. We learned to design with sensitivity, particularly given the mental health focus of the app.
Accessibility and Inclusivity: Incorporating features like speech interaction and anonymous login emphasized our commitment to making the app accessible and comfortable for all users.

In Summary

Our journey with Earlent was not just about building an application but also about growing as developers and as a team. We expanded our technical skills across various platforms and services, a better understanding of user-centric design, and reinforced our commitment to ethical considerations in technology.

What's Next for Earlent?

We are committed to continually enhancing Earlent to better serve our users' mental health needs. Here are some of the exciting features and improvements we have planned for future updates:

1. Community Forum

We plan to introduce a community forum similar to Reddit, where users can connect with others, share experiences, and offer support. This platform will foster a sense of community, allowing users to engage in discussions, ask questions, and provide mutual encouragement in a safe and moderated environment.

2. Customizable 3D Avatar with Multilingual Capabilities

We aim to implement a customizable 3D avatar that interacts with users within the app. Users will be able to personalize the avatar's appearance and choose from a variety of voices and languages. This feature will enhance user engagement by providing a more immersive and personalized experience.

3. AI-Driven Data Management

Users will soon be able to manage their data directly through the AI chat interface. This includes scheduling, updating, or deleting check-ins and other personal data by simply communicating with the AI. This feature will streamline data management, making it more intuitive and user-friendly.

4. User-Doctor Communication Bridge

We plan to develop a secure communication bridge between users and mental health professionals. This feature will allow users to connect with licensed therapists and doctors directly through the app, facilitating access to professional support when needed.

5. Multilingual Support

To make Earlent accessible to a broader audience and promote diversity, we will introduce support for multiple languages. Users will be able to interact with the AI and navigate the app in their preferred language, breaking down language barriers and reaching a global user base.

6. Enhanced Agentic Workflow and Personalization

We are working on further enhancing our agentic workflow to allow the AI model to keep track of users' progress over time. This will enable the AI to provide more personalized support and insights, which will be presented to users through an interactive dashboard. This feature will help users visualize their mental health journey and identify areas of improvement.

7. Knowledge Base and Favorites

We plan to build a comprehensive knowledge base of mental health resources that the AI can refer to when assisting users. Additionally, users will have the ability to "favorite" resources, allowing for easy access to preferred articles, exercises, or recommendations in the future.

8. Expanded Voice Options for the AI

To enhance personalization, we will introduce a variety of voice options for the AI, allowing users to choose the voice that they find most comforting and relatable. This includes different accents, tones, and speech patterns to cater to individual preferences.

By focusing on these developments, we aim to make Earlent an even more powerful and versatile tool for mental health support. Our goal is to create a platform that not only provides empathetic assistance but also fosters a supportive community, offers professional connections, and adapts to the unique needs of each user.