PDF Pal: A Modern RAG-Powered Chatbot for Your Documents

Author: Chidambara Raju G
Version: 1.0
Project Repository: PDF Pal

---

🚀 Overview

PDF Pal is a powerful, intuitive, and high-performance chatbot application designed to transform your static PDF documents into dynamic conversational partners. Built with Streamlit and powered by the cutting-edge Retrieval-Augmented Generation (RAG) architecture, PDF Pal allows you to "talk" to your documents. Simply upload one or more PDFs, and ask questions in plain English to get concise, context-aware answers instantly.

This project is perfect for students, researchers, legal professionals, and anyone who needs to quickly extract information from dense documents without manually searching page by page. By leveraging the speed of Groq's Llama 3.1 model and the efficiency of local embeddings, PDF Pal delivers a seamless and responsive user experience.

---

✨ Key Features

• Multi-PDF Support: Upload and chat with multiple PDF documents simultaneously, creating a unified knowledge base.
• Conversational Memory: The chatbot remembers previous turns in the conversation, allowing for follow-up questions and more natural interactions.
• High-Speed Responses: Utilizes the Groq API for near-instantaneous inference with the powerful Llama 3.1 70B model.
• Efficient & Private: Employs state-of-the-art local sentence transformers for embedding, ensuring your document content can be processed without leaving your machine.
• Simple & Clean UI: A user-friendly interface built with Streamlit makes the application accessible to everyone, regardless of technical skill.
• Modern RAG Implementation: Uses a sophisticated LangChain pipeline that incorporates a history-aware retriever for more accurate and contextually relevant results.

---

Retrieval-Augmented Generation (RAG) is a sophisticated architecture that enhances the capabilities of Large Language Models (LLMs) by grounding them in external knowledge. Think of it as giving an LLM an open-book exam. Instead of relying solely on its pre-trained (and potentially outdated) knowledge, the model can first retrieve relevant information from your specific documents and then use that information to generate a well-informed answer.

The PDF Pal application implements a modern, conversational RAG pipeline. The process is broken down into two main phases: 1. Indexing (processing the documents) and 2. Retrieval & Generation (answering questions).

Phase 1: Indexing (The "Processing" Step)

This happens when you upload your PDFs and click the "Process" button. The goal is to convert your documents into a searchable knowledge base.

1. Document Ingestion: The application first reads your uploaded PDF files using the PyPDF2 library. The get_pdf_text function extracts all the raw text from every page of every document you provide.

2. Text Splitting (Chunking): LLMs have a limited context window (the amount of text they can consider at one time). A large document cannot be fed to the model all at once. Therefore, the extracted text is split into smaller, manageable "chunks" using RecursiveCharacterTextSplitter from LangChain. This method intelligently splits text by paragraphs, sentences, and words to keep related content together. The chunk_overlap parameter ensures that context is not lost at the boundaries of chunks.

3. Embedding: The text chunks are then converted into numerical representations called embeddings or vectors. Each vector captures the semantic meaning of the text chunk. This is the most crucial step for enabling semantic search. PDF Pal uses the HuggingFaceEmbeddings library with the highly efficient sentence-transformers/all-MiniLM-L6-v2 model, which runs locally on your machine.

4. Vector Storage: These embeddings are stored and indexed in a vector database. PDF Pal uses FAISS (Facebook AI Similarity Search), which is an extremely fast, in-memory library for searching through millions of vectors to find the ones most similar to a query vector. This indexed collection of vectors is our vectorstore.

Phase 2: Retrieval & Generation (The Chat Step)

This phase occurs every time you ask a question.

5. History-Aware Query Formulation: This is a key feature of PDF Pal's modern RAG design. When you ask a follow-up question like "What about its impact on the economy?", the model needs context from the chat history. The create_history_aware_retriever does exactly this. It first takes your latest question and the chat history, and asks the LLM to rephrase it into a standalone question. For example, if the previous topic was "the industrial revolution," your follow-up might be reformulated into "What was the industrial revolution's impact on the economy?".

6. Semantic Retrieval: The standalone question is then converted into an embedding. This query embedding is used to perform a similarity search in the FAISS vector store. The retriever fetches the top 'k' most relevant text chunks from your original documents whose embeddings are closest to the query's embedding.

7. Augmentation & Generation: The retrieved chunks (the "context") are then "stuffed" into a prompt along with the original question and the chat history. This final, augmented prompt is sent to the ChatGroq LLM (llama-3.1-70b-versatile). The prompt essentially says: "Using our chat history and the following retrieved context from the documents, answer this question."

8. Final Answer: The LLM generates a response based only on the provided context. This prevents hallucination (making up answers) and ensures the answer is grounded in the source documents. This generated answer is then displayed to you, and the conversation is saved to continue the cycle.

---

🛠️ Technology Stack

• Application Framework: Streamlit
• Orchestration: LangChain
• LLM: Groq (Llama 3.1 70B)
• PDF Processing: PyPDF2
• Embeddings: Hugging Face Sentence Transformers (all-MiniLM-L6-v2)
• Vector Store: FAISS (Facebook AI Similarity Search)
• Environment Management: python-dotenv

---

⚙️ Setup and Installation

To run this project locally, follow these steps:

1. Clone the Repository

   git clone https://github.com/ChidambaraRaju/pdf-pal-rag-document-assistant
   cd pdf-pal-rag-document-assistant

2. Create a Virtual Environment

   python -m venv venv
   source venv/bin/activate  # On Windows, use `venv\Scripts\activate`

3. Install Dependencies

   pip install -r requirements.txt

   (Note: You'll need to create a requirements.txt file containing streamlit, langchain, pypdf2, faiss-cpu, sentence-transformers, langchain-groq, python-dotenv, etc.)

4. Set Up Environment Variables
   Create a file named .env in the root directory and add your Groq API key:

   GROQ_API_KEY="your_groq_api_key_here"
   

5. Run the Application

   streamlit run app.py

   The application will open in your web browser.

---

📄 Source Code

Here is the complete source code for the application (app.py):

import streamlit as st
import os
from dotenv import load_dotenv
from PyPDF2 import PdfReader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain_groq import ChatGroq
from langchain.chains.retrieval import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains.history_aware_retriever import create_history_aware_retriever
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage, AIMessage

#************************************ Helper Functions ******************************************

def get_pdf_text(pdf_docs):
    """Extracts text from a list of uploaded PDF documents."""
    text = ""
    for pdf in pdf_docs:
        pdf_reader = PdfReader(pdf)
        for page in pdf_reader.pages:
            text += page.extract_text()
    return text

def get_text_chunks(text):
    """Splits the text into smaller chunks"""
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
        length_function=len
    )
    text_chunks = text_splitter.split_text(text)
    return text_chunks

def get_vectorstore(text_chunks):
    """Creates a vector store from the text chunks"""
    embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
    vector_Store = FAISS.from_texts(texts=text_chunks, embedding=embeddings)
    return vector_Store

def get_retrieval_chain(vector_store):
    """Creates the main retrieval chain"""
    llm = ChatGroq(model="llama-3.1-70b-versatile", temperature=0.1)
    retriever = vector_store.as_retriever()
    
    contextualize_q_system_prompt = """Given a chat history and the latest user question which might reference context
    in the chat history, formulate a standalone question which can be understood without the chat history. DO NOT answer 
    the question. Just formulate if it is needed otherwise return it as it is.
    """
    
    contextualize_q_prompt = ChatPromptTemplate.from_messages([
        ("system", contextualize_q_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}")
    ])
    history_aware_retriever = create_history_aware_retriever(llm=llm, retriever=retriever, prompt=contextualize_q_prompt)
    
    qa_system_prompt = """You are an assistant for question-answering tasks. Use the following pieces of context to answer the question at the end.
    If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum and keep the answer as concise as possible.
    Context: {context}
    """
    qa_prompt = ChatPromptTemplate.from_messages([
        ("system", qa_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}")
    ])
    
    question_answer_chain = create_stuff_documents_chain(llm=llm, prompt=qa_prompt)
    
    rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)
    return rag_chain

def handle_userinput(user_question):
    """Handles user input and the conversation flow."""
    if st.session_state.retrieval_chain is None:
        st.warning("Please upload and process your documents before asking a question.")
        return

    response = st.session_state.retrieval_chain.invoke({
        "chat_history": st.session_state.chat_history,
        "input": user_question
    })
    
    st.session_state.chat_history.append(HumanMessage(content=user_question))
    st.session_state.chat_history.append(AIMessage(content=response["answer"]))

    # Display chat history
    for i, message in enumerate(st.session_state.chat_history):
        if isinstance(message, HumanMessage):
            st.write(f"**You:** {message.content}")
        elif isinstance(message, AIMessage):
            st.write(f"**Bot:** {message.content}")

#*************************************************** Streamlit App ***************************************

def main():
    load_dotenv()
    os.environ["GROQ_API_KEY"] = os.getenv("GROQ_API_KEY")
    st.set_page_config(page_title="Chat with your PDFs", page_icon=":books:")

    # Initialize session state variables
    if "retrieval_chain" not in st.session_state:
        st.session_state.retrieval_chain = None
    if "chat_history" not in st.session_state:
        st.session_state.chat_history = []

    st.header("Chat with your PDFs (Modern RAG 🚀)")
    user_question = st.text_input("Ask a question about your documents:")
    if user_question:
        handle_userinput(user_question)

    with st.sidebar:
        st.subheader("Your documents")
        pdf_docs = st.file_uploader(
            "Upload your PDFs here and click on 'Process'", accept_multiple_files=True)
        if st.button("Process"):
            if pdf_docs:
                with st.spinner("Processing..."):
                    raw_text = get_pdf_text(pdf_docs)
                    text_chunks = get_text_chunks(raw_text)
                    vectorstore = get_vectorstore(text_chunks)
                    
                    # Create and store the chain in session state
                    st.session_state.retrieval_chain = get_retrieval_chain(vectorstore)
                    st.session_state.chat_history = [] # Reset history on new processing
                    st.success("Done!")
            else:
                st.warning("Please upload at least one PDF file.")

if __name__ == '__main__':
    main()

📊 Performance Evaluation

System-Level Metrics

---

Retrieval Quality Metrics

---

🔮 Future Work

• Support for More Document Types: Extend functionality to support .docx, .txt, and URLs.
• Persistent Vector Stores: Integrate with persistent vector databases like ChromaDB or Pinecone to avoid reprocessing files on each app restart.
• Source Highlighting: Display the specific text chunks from the source documents that were used to generate the answer.
• UI Enhancements: Add features like a "Clear Chat" button and user feedback options.