RAG Assistant =Talking to Your Documents using Artificial Intellegence

Introduction
Have you ever wished you could just ask your research paper a question and get a clear answer, instead of scrolling through pages of dense text?

That’s exactly what my RAG Assistant does.

It’s a Retrieval-Augmented Generation (RAG) system built using LangChain, Hugging Face, and ChromaDB —
designed to help you talk directly to your own PDFs.

For this project, I used the legendary “Attention Is All You Need” paper — the one that introduced Transformers (the foundation of GPT, BERT, and modern AI).
Now, instead of re-reading the 10-page paper every time, I can simply ask:
“What is self-attention?”
and my assistant instantly replies with a clear explanation pulled from the actual paper.

---

Why I Built This
Most AI assistants give generic answers.
They “sound right,” but you can’t trust them on specific documents.

So I wanted to build something that:

• Understands your own files
• Retrieves exact context from them
• And then answers intelligently — all locally, without cloud dependencies.

This project became my first true hands-on experience in Agentic AI development,
where I learned how AI models retrieve, reason, and respond.

---

How It Works — The Magic Under the Hood
Here’s what happens behind the scenes (in simple terms):

1. Document Loading
   I drop my PDFs (like research papers) inside the docs/ folder. The system reads them using:
   python
   from langchain.document_loaders import PyMuPDFLoader

2. Text Splitting
   Big documents are split into smaller, readable chunks:
   RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=100)
   This ensures the model doesn’t get overwhelmed.

3.Embedding and Storage
Each text chunk is converted into a numerical vector (embedding) using Sentence Transformers (MiniLM-L3-v2) —and stored inside a Chroma database for fast retrieval.

4. Question Time
   When you ask something, the retriever finds the top 3 most relevant chunks based on meaning — not keywords.

5. Answer Generation
   The question + retrieved context is passed to a Flan-T5 model, which generates a final, context-based answer in plain English.

6. Chat Interface
   You can ask questions interactively right inside Jupyter Notebook.

---

Example Interaction

You:

What is multi-head attention?

RAG Assistant:

Multi-head attention allows the model to focus on information from different representation subspaces at different positions.
It helps capture multiple relationships in parallel.

---

Tech Stack
Layer Tool / Library
Document Loader LangChain (PyMuPDFLoader)
Text Splitter RecursiveCharacterTextSplitter
Embeddings Sentence Transformers (MiniLM-L3-v2)
Vector Store Chroma
Language Model Google Flan-T5 Base
Framework LangChain
Interface Jupyter Notebook

---

Project Structure
rag-assistant/
│
├── rag_assistant.ipynb # Main notebook
├── README.md # Overview + setup guide
├── docs/ # Custom PDFs
│ └── attention-is-all-you-need-Paper.pdf
└── chroma_db/ # Vector store files

---

How to Run

Step 1: Install dependencies

pip install langchain chromadb sentence-transformers pymupdf tqdm

Step 2: Place your PDFs in the docs/ folder

Step 3: Run the notebook cell by cell

jupyter notebook rag_assistant.ipynb

Step 4: Ask anything!
Try:

“What are positional encodings?”

“How does self-attention work?”

“What datasets were used to train the Transformer?”

Output Example

Input:

What is the scaled dot-product attention formula?

Output:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ) V

Why It’s Special

Works 100% locally — no external API calls

Uses modern open-source AI tools

Easy to customize for any topic (replace PDFs)

Transparent and explainable — every step is visible

Great foundation for building a chatbot, research assistant, or document explorer

Future Ideas

I’m planning to extend this project by:

Adding memory for multi-turn conversations

Creating a Streamlit-based web interface

Adding support for DOCX and web articles

Improving retrieval quality using hybrid search

GitHub Repository https://github.com/Diya191110/rag-assistant

RAG Assistant (LangChain + Chroma + Flan-T5)

About the Author

Diya Panda
B.Tech CSE, KIIT University
Agentic AI Developer Certification — Module 1 Project

I’m passionate about AI that’s practical, explainable, and human-centered .This project taught me how retrieval and reasoning come together —and it’s just the beginning of my journey into Agentic AI development.

Conclusion

The RAG Assistant turns static PDFs into interactive knowledge companions.Instead of reading through pages, you can now talk to your documents. It’s a small step towards AI systems that truly understand our world —
one question at a time.