My First RAG Assistant: Document QA System with Groq AI

🎯 Project Overview

I built a Retrieval-Augmented Generation (RAG) assistant that answers questions from custom documents using Groq AI, ChromaDB, and Sentence Transformers. This project demonstrates the core concepts of Agentic AI by implementing a complete document QA pipeline.

🛠️ Technology Stack

• LLM: Groq AI (Llama-3.1-8b-instant)
• Vector Database: ChromaDB
• Embeddings: Sentence Transformers (all-MiniLM-L6-v2)
• Framework: LangChain-style implementation (custom)
• Environment: Python with local storage

🔧 How It Works

My implementation follows these steps:

1. Document Processing: Loads JSON documents and chunks them for efficient retrieval
2. Vectorization: Creates embeddings using Sentence Transformers
3. Storage: Saves vectors in ChromaDB for fast similarity search
4. Retrieval: Finds relevant document chunks for user queries
5. Generation: Uses Groq AI to generate answers based on retrieved context

📁 Code Structure

my-rag-project/
├── rag_pipeline.py # Main RAG orchestration
├── vector_store.py # Vector database management
├── json_processor.py # Document loading & chunking
├── test_llm.py # API connection test
├── requirements.txt # Dependencies
└── README.md # Project documentation

🚀 Key Features Implemented

• ✅ Custom Document Ingestion: Supports JSON format with automatic chunking
• ✅ Vector Search: Implements semantic search using cosine similarity
• ✅ RAG Pipeline: Full retrieval → augmentation → generation flow
• ✅ Local Storage: ChromaDB runs locally without external services
• ✅ Free Tier: Uses Groq's free API for cost-effective development

🎮 Demo Results

🧠 Learning Outcomes

Through this project, I learned:

1. How to implement a complete RAG pipeline from scratch

2. The importance of proper document chunking for retrieval quality

3. How vector databases enable semantic search

4. Prompt engineering for better answer generation

5. Error handling in AI pipelines

🔗 GitHub Repository

RAGPipeline

🔄 Future Enhancements

Add conversation memory for multi-turn dialogues

Support additional document formats (PDF, DOCX)

Implement ReAct pattern for complex reasoning

Add web interface using Streamlit

Deploy as a REST API service

📚 Resources Used

Ready Tensor AAIDC Module 1 materials

Groq API documentation

ChromaDB documentation

Sentence Transformers models

This project was completed as part of Module 1 of the Agentic AI Developer Certification Program.