Abstract
The rapid adoption of medical devices among patients has highlighted a critical gap in their effective usage due to inexperience and lack of guidance. This app addresses this issue by combining generative AI with advanced technologies to deliver intuitive, user-friendly solutions. Designed with a focus on India’s healthcare challenges, the app empowers users by providing personalized medical device guides, medication adherence tracking, and insightful analytics.
The app employs Streamlit for an interactive user interface and integrates Gemini LLM to generate detailed step-by-step guides tailored to each user’s medical history. Leveraging the YouTube API, users can access relevant video tutorials for visual guidance, while a YouTube summarization feature—powered by Deepgram AI and Qwen LLM—offers concise insights for users with limited time.
The future idea for the app is to enhance medication adherence, the app sends personalized reminders and tracks responses, utilizing NLP techniques to analyze reasons for missed medications. Weekly adherence reports are shared with emergency contacts, ensuring accountability and support, integrating doctors and hospitals for seamless appointments and creating a robust authentication system for personalized notifications.
Wanna give it a try? : https://svaisthi.streamlit.app/
you can access the code at https://github.com/Masterhazi/svAIsthi
The main problem we faced whenever a patient was handed a medical device was their inexperience and lack of clarity on how to use it. I wanted to solve this issue in a way that users could clarify their doubts instantly, while also getting video guidance from YouTube’s extensive library. Additionally, there are several challenges in the Indian medical system, like the lack of proper tracking for medications, allergies, or adverse reactions. Unless a patient keeps seeing the same doctor throughout their life, this critical information is often lost. This motivated me to create a solution that raises awareness about medication interactions and adverse reactions.
Another big issue was medication adherence—patients miss medications frequently, with no accountability. Their loved ones often have no idea, and people are too busy to update tracker apps after taking their meds. I wanted to address this by creating a system that not only tracks adherence but also sends notifications and provides weekly insights into their medication practices. These insights help users better understand their routines and stay on top of their health.
This project began with a Streamlit UI and Gemini LLM to generate a guide whenever a medical device is shown. Using a file uploader in Streamlit, users could upload images of medical devices, and the app would generate a personalized guide. Basic user data—such as name, age, previous medications, current medications, and health conditions—was collected to make the guidance more relevant and tailored.
The prompt for Gemini was refined to ensure it provided awareness about medication adherence, drug interactions, and side effects in a way that users of all ages could easily understand. The LLM also focused on simplifying the guidance so users could follow the steps anytime they needed.
A major improvement came when I realized that users might not always want to read a detailed guide, especially if they’re short on time. That’s where the visual guide feature came in. Using YouTube’s massive video library, I designed the app to fetch tutorial videos for medical devices. With the YouTube API, the app searched for relevant videos based on the titles generated by the LLM. These videos were then embedded in the app using st.video, so users could choose between a textual guide or a video guide, depending on their preference.
To make the app more engaging and accessible, I also introduced some visual enhancements to the interface.
With everything running smoothly, I noticed another potential problem—what if users didn’t have time to watch a video and wanted a quick summary instead? This led to the addition of a YouTube summarization feature. By integrating yt-dl for downloading audio, Deepgram AI for transcriptions, and the Qwen LLM model from Hugging Face for summarization, the app could now provide concise, step-by-step summaries of YouTube videos.
This made the app even more useful, ensuring users had access to quick and actionable insights without needing to watch an entire video or read lengthy text.
Future enhancements planned for the application include:
Medication Notifications with Contextual Tracking:
Notifications will prompt users at the time of medication, with options to confirm (Yes/No). If the user selects "Yes," the app records compliance in the database. If "No," the app prompts the user for a reason, allowing input through voice or text. The system will use NLP and cosine similarity to match user-provided reasons to previously entered ones, enabling streamlined data analysis.
Repeated reasons will be displayed for quick selection during future missed medication entries.
Emergency Contact Alerts:
Weekly adherence insights will be emailed to the user’s emergency contacts. If a user fails to respond to a medication notification, the device will vibrate for 5 minutes. In the absence of any response, an alert message will be sent to emergency contacts.
Doctor and Hospital Integration:
The application will integrate doctors and hospitals into the platform, easing appointment processes. To ensure affordability, no commissions will be taken from doctors, keeping service costs as close to original prices as possible.
User Authentication and Database Customization:
Robust user authentication and a personalized database system are under development. These will allow tailored notifications and customizations for individual users, ensuring a seamless and secure experience.
The app now provides comprehensive support, including:
The application ensures that users can confidently navigate their medical devices, adhere to medication routines, and gain valuable insights to manage their health effectively.
There are no datasets linked
There are no datasets linked
