MEDLLM

Shreyash Pingle developed a Retrieval-Augmented Generation (RAG) chatbot designed to efficiently store and retrieve medical data. The chatbot can handle large-scale medical documents by leveraging Pinecone as a vector database, enabling quick and accurate responses. This system is particularly beneficial for professionals and researchers who need instant access to specific information from extensive medical texts.

Features

Efficient Data Handling: Processed and indexed 600+ pages of medical documents for seamless retrieval.
Advanced NLP Model: Integrated sentence-transformers/all-MiniLM-L6-v2 to enhance natural language understanding.
Optimized Query Processing: Used LangChain to streamline data retrieval and improve chatbot responses.
User-Friendly Interface: Built an interactive Flask-based UI with HTML and CSS for smooth chatbot interactions.

CODE SNIPPETS

Splitting data into chunks

def text_split(extracted_data):
    text_splitter=RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=20)
    text_chunks=text_splitter.split_documents(extracted_data)
    return text_chunks

API keys

os.environ["PINECONE_API_KEY"] = PINECONE_API_KEY
os.environ["GROQ_API_KEY"] = GROQ_API_KEY

Invoking function

response = rag_chain.invoke({"input": "What is treatment for ablyopia?"})
print(response["answer"])

Storing Indexes in Pinecone

extracted_data=load_pdf_file(data='Data/')
text_chunks=text_split(extracted_data)
embeddings = download_hugging_face_embeddings()


pc = Pinecone(api_key=PINECONE_API_KEY)

index_name = "medicalbot"


pc.create_index(
    name=index_name,
    dimension=384, 
    metric="cosine", 
    spec=ServerlessSpec(
        cloud="aws", 
        region="us-east-1"
    ) 
) 

# Embed each chunk and upsert the embeddings into your Pinecone index.
docsearch = PineconeVectorStore.from_documents(
    documents=text_chunks,
    index_name=index_name,
    embedding=embeddings, 
)

#Conclusion
This project showcases the power of RAG-based AI chatbots in improving accessibility to medical knowledge. By combining vector search, NLP models, and an intuitive UI, the chatbot efficiently extracts relevant insights from large datasets. This solution can be extended to other domains requiring fast, context-aware information retrieval, making it a valuable tool for research and professional use.