RAG Assistant: Local Document Retrieval with LLMs

A Retrieval-Augmented Generation (RAG) assistant that answers user queries using a collection of local documents. It retrieves relevant text chunks from your dataset and generates answers using LLMs.

---

Motivation

Many AI applications require answering questions based on domain-specific documents. RAG combines:

1. Retrieval: finding relevant information from documents.
2. Generation: producing coherent answers using a large language model (LLM).

This project demonstrates a simple, modular RAG assistant using local text files.

---

Features

• Retrieves relevant chunks from .txt documents using embeddings.
• Supports multiple LLM backends:
  • OpenAI (gpt-3.5-turbo, gpt-4o-mini)
  • Groq LLaMA 3.1 8B Instant
  • Google Gemini API
• Handles document metadata and context tracking.
• Works offline with local embeddings.

---

Method / Architecture

The workflow:

User Query
    │
    ▼
Retrieve relevant document chunks
    │
    ▼
Combine context
    │
    ▼
Send to LLM (OpenAI / Groq / Google Gemini)
    │
    ▼
Return generated answer

1. Document ingestion: Reads .txt files from a content directory.
2. Chunking & embeddings: Splits documents into manageable chunks and generates embeddings.
3. Vector search: Finds the top n_results relevant chunks per query.
4. LLM generation: Sends the combined context and query to the selected LLM for response.

---

Supported LLM Backends

Recommended: manage quotas carefully or use local Groq models for higher throughput.

---

Usage

from rag_assistant import RAGAssistant

rag = RAGAssistant()

while True:
    query = input("Enter a question or 'quit' to exit: ")
    if query.lower() == "quit":
        break
    result = rag.query(query)
    print("Answer:", result["answer"])

Example Query

Enter a question or 'quit' to exit: What is an asteroid?
Answer: An asteroid is a small rocky body orbiting the Sun, primarily found in the asteroid belt between Mars and Jupiter.

---

Notes / Tips

• Document loading now joins the content directory with filenames to avoid FileNotFound errors.
• For vague queries (like "test"), consider skipping retrieval or returning a generic response.
• Adjust n_results to limit irrelevant chunks.
• Embedding quality directly affects answer relevance.

---

Dependencies

• Python 3.10+
• openai (for OpenAI API)
• chromadb or other vector database backend (for Groq / embeddings)
• sentence-transformers (for embeddings)
• python-dotenv (for .env loading)
• langchain (optional, for prompt chaining)

Install dependencies:

pip install openai chromadb sentence-transformers python-dotenv langchain

---

Future Work

• Add more local datasets for testing.
• Support additional LLMs or embeddings backends.
• Improve query relevance with better chunking and similarity thresholds.

---

License

MIT License