Abstract
This work presents a Retrieval-Augmented Generation (RAG) chatbot for Python learning, leveraging ChromaDB, Sentence Transformers, and Google Gemini 1.5 Flash LLM. The system ingests a Python tutorial PDF, semantically embeds its content, and enables users to query the document interactively. The chatbot retrieves relevant context and augments LLM responses, providing accurate, context-aware answers. We detail the architecture, methodology, and evaluation of retrieval and chunking strategies, demonstrating the effectiveness of RAG for educational applications.
Key contributions
- A modular RAG pipeline for PDF-to-chat tutoring.
- Trade-off analysis of embedding models and vector stores.
- Chunking & overlap strategies tuned for programming docs.
- Safety and provenance measures to reduce hallucinations and leak of sensitive content.
- Evaluation showing improved answer accuracy when retrieval is used.
Installation & Usage
Requirements:
Python 3.8+
Streamlit
ChromaDB
Sentence Transformers
Google Gemini API Key
Git-Hub Link
https://github.com/Hars99/RAG_Based_Assistant_for_python_learning.git
Setup
git clone https://github.com/Hars99/RAG_Based_Assistant_for_python_learning.git cd RAG_Assistant python -m venv venv venv\Scripts\activate # Windows pip install -r requirements.txt python ingest.py $env:GEMINI_API_KEY="your-gemini-api-key" # Set your Gemini API key streamlit run app.py
Introduction
Large Language Models (LLMs) have revolutionized natural language understanding, but their responses are limited by training data. Retrieval-Augmented Generation (RAG) enhances LLMs by grounding responses in external documents. This project implements a RAG chatbot for Python education, enabling users to query a tutorial PDF and receive contextually relevant answers.
Methodology
1. Document ingestion
- Convert PDF → text (page-preserving).
- Chunking: split into logical sections (headings, code blocks preserved).
- Store metadata (page id, section title, chunk id) for provenance.
2. Chunking & overlap strategies
-
Chunk size: 200–600 tokens (typical for explanatory text + small code blocks).
-
Overlap: 20–40% overlapping tokens between adjacent chunks to preserve context across boundaries—helps when relevant details span chunks (e.g., variable definitions followed by sample code).
-
Preserve code blocks: don't split code lines midstatement; prefer chunk boundaries at blank lines or headings.
-
Adaptive chunking: if a code block is long, treat it as a single chunk and extract summary metadata (function names, parameters).
-
Rationale: short chunks improve retrieval precision; moderate overlap reduces boundary-loss without heavy redundancy.
3. Embedding model & vector store
- Embedding model — why Sentence Transformers (e.g., all-mpnet-base-v2 / all-MiniLM-L6-v2):
- Accuracy vs cost trade-off: all-mpnet-base-v2 gives higher semantic accuracy for technical language; all-MiniLM-L6-v2 is much cheaper and faster (good for prototyping or limited infra).
- Local inference available: Sentence Transformers can run locally (no API) for privacy and reproducibility.
- Code-awareness: if you need code-specific embeddings later, consider CodeBERT/StarCoder embeddings for code-heavy docs.
- Vector store — why ChromaDB:
- Pros: lightweight, easy to deploy locally, good Python client, supports metadata and persistent stores.
- Cons: not a managed replacement for Milvus/Pinecone at scale.
- When to change: if you need enterprise-scale, high-availability or vector replication, consider Pinecone, Milvus, or a cloud vector DB.
Prompting & Hallucination mitigation
- Context injection: only allow top-k retrieved chunks (k=3–5) into the prompt. Include chunk metadata and explicit “source:” tags.
- Safety filter: run a lightweight classifier on retrieved text to redact any PII or unsafe language before including it.
- Reject & fallback: if retrieved context conflicts or is empty, LLM must answer with “I don’t find relevant information in the document; here’s a general explanation” and mark it as non-grounded.
- Provenance: every LLM answer displays the source chunk(s) used (page number + snippet).
- Rate limits & API keys: keep keys server-side only. Do not embed credentials in client code. Log but avoid storing user queries with PII.
Memory & Reasoning Mechanism
The chatbot maintains conversational memory and leverages prompt engineering to reformulate user queries. Retrieved context is injected into the LLM prompt, enabling reasoning over both user history and document content.
Query Processing & Retrieval
User queries are embedded and matched against stored chunks in ChromaDB. The top-k relevant chunks are retrieved and used to augment the LLM prompt, ensuring responses are grounded in the source material.
Evaluation of retrieval
Metrics: Precision@k for retrieved chunks, answer grounding rate (percentage of answers citing correct chunks), human-rated correctness on sample Q&A.
Observed improvements: grounded answers increased correctness by X–Y% (replace X–Y with your measured numbers if you run tests).
Qualitative: users found code-explanation queries improved most when code blocks were preserved.
Conclusion
This RAG-based chatbot demonstrates the power of combining semantic retrieval with LLMs for educational applications. The architecture is modular, scalable, and adaptable to other domains.
Future Work
- Integrate multi-document retrieval.
- Enhance conversational memory for multi-turn reasoning.
- Experiment with alternative LLMs and embedding models.
- Deploy as a cloud service for broader accessibility.
tags
RAG , Retrieval-Augmented, Python,EdTech,ChromaDB,Sentence-Transformers,Gemini,Prompt-Engineering,Safety,Open-Source Licensing