This pilot study introduces Dr. Phil, a family doctor AI agent, which has been implemented across three distinct AI agent frameworks to evaluate its performance, adaptability, and user experience. Dr. Phil is designed to provide medical advice, diagnose symptoms, and offer health recommendations based on user inputs, with features such as lab results explanation and suggestion, physical examination report explanation and suggestion, recommendation to and connection with specialist physicians, with long term memory. By leveraging different AI agent frameworks such as Dify, ElizaOS, Baidu AppBuilder, this study aims to explore the strengths and weaknesses of each platform in accommodating such a specialized AI application.
There are many AI agent frameworks/libraries emerging with rapid advancement and development of AI large models with tool uses. We choose three AI agent frameworks in diverse spectrum, either open source or proprietary, web2 or web3, workflow or agentic, full code or low code: Dify, ElizaOS, Baidu AppBuilder.
Dify is an open-source platform designed for building AI applications, particularly those involving generative AI and large language models (LLMs). It provides developers with tools to create AI agents, supporting multiple LLMs, advanced prompt engineering, and Retrieval Augmented Generation (RAG) engines. Its flexible agent framework makes it suitable for various applications, potentially including medical AI agents like Dr. Phil, due to its customization options and community support. The platform's open-source nature encourages community contributions, which could be beneficial for customizing applications to specific domains, such as healthcare.
The platform's documentation, available at Dify Documentation, highlights its use by over 180,000 developers and 59,000+ end users, suggesting a robust community and ecosystem. While there is no specific mention of medical applications, its general-purpose nature and customization options make it a promising choice for implementing a medical AI agent like Dr. Phil. The open-source aspect could also facilitate compliance with healthcare regulations by allowing modifications to ensure data privacy and security.
ElizaOS is an open-source framework for building autonomous AI agents, known for enabling interactions across platforms like Discord, X (Twitter), and Telegram. It features memory management, media processing, and web3 integration, which could be useful for creating AI agents with consistent personalities and diverse engagement capabilities, possibly applicable to healthcare scenarios.
Information from ElizaOS Introduction and ElizaOS GitHub indicates a TypeScript-based implementation with a modular architecture, making it highly extensible. However, there is no specific mention of its use in medical contexts, suggesting that its application to Dr. Phil would require additional customization to meet healthcare standards, such as HIPAA compliance in certain regions.
Baidu AppBuilder, part of Baidu AI Cloud, is a set of tools for creating AI-native applications. It allows developers to build applications using natural language, offering frameworks for retrieval-augmented generation (RAG), agents, and generative BI. Given Baidu's involvement in healthcare AI, it seems support medical AI agent development, such as Dr. Phil, with its user-friendly interface.
AppBuilder seems well-positioned for medical AI agent development. Key features relevant to Dr. Phil include:
Natural language-based development, which could simplify the creation of medical AI agents, potentially reducing development time and costs.
RAG frameworks, which could enhance the agent's ability to retrieve and integrate medical knowledge, improving the accuracy of diagnoses and recommendations.
Integration with Baidu's AI ecosystem, which may provide access to healthcare-specific models and datasets, given Baidu's prior work in areas like cancer detection and patient triage.
To organize the comparison, we present a table summarizing key features of each framework and their potential relevance to the pilot study:
This table highlights that each framework brings unique strengths to the table, with Dify offering flexibility and community support, ElizaOS providing multi-platform capabilities, and Baidu AppBuilder leveraging Baidu's healthcare AI experience. The pilot study's focus on performance, adaptability, and user experience will likely reveal how these features translate into practical outcomes for Dr. Phil, such as the accuracy of medical advice, ease of integration with healthcare systems, and patient satisfaction.
We programmed Dr. Phil agent in each of the three frameworks. For Dify, workflow-like agent is built according to Dify workflow builder. For ElizaOS, Dr. Phil is pragrammed as a specialized character with medical knowledge base. For Baidu AppBuilder, Dr. Phil is designed as a wellness guardian specialized with traditional chinese medicine, natural therapy, with modern medical knowledge, speaking in Chinese, catering to Chinese-speaking patients/customers.
We did extensive experiments chatting with Dr. Phil, in both English and Chinese. Tested with lab result explanation, physical exam report explanation, with long-term memory pickup testing. Dr. Phil(Wellness Guardian) answers with both modern medical diagnosis and suggestion, and traditional chinese medicine diagnosis and suggestion. For better chinese language generation and traditional chinese medicine result, the models used are LLM DeepSeek R1 for planning/thinking and multi-modal model ERNIE4.5 for text generation and VL understanding and generation. It shows that Dr. Phil(Wellness Guardian) designed with Baidu AppBuilder is well suited to Chinese-speaking patients. A video demo is attached for reference.
The results were particularly impressive when it came to handling medical terminology and providing clear, concise explanations. Users appreciated the combination of Dr. Phil's expertise and the intuitive interface provided, which made navigating through complex medical information seemless. Furthermore, the long-term memory pickup testing revealed that patients were able to retain and understand the information presented by Dr. Phil(Wellness Guardian) significantly better compared to traditional methods. The video demo showcases these capabilities vividly, highlighting the seamless interaction and the effectiveness of the wellness assistant in a real-world scenario, capable of commercialization.
The pilot study's use of Dify, ElizaOS, and Baidu AppBuilder reflects a diverse approach to evaluating AI agent frameworks for a specialized medical application. While Dify's open-source nature and ElizaOS's multi-platform capabilities offer flexibility and engagement, Baidu AppBuilder's integration with Baidu's healthcare initiatives could provide a competitive edge in medical contexts. Future research might explore how these frameworks handle specific healthcare challenges, such as data security, regulatory compliance, and scalability, to determine their long-term viability for medical AI agents like Dr. Phil.