This project uses a vector database to index and search information in text documents, then generates intelligent responses based on the retrieved content using Large Language Models (LLMs).
Key Features

📚 Document Processing: Automatically loads and processes text documents
🔍 Semantic Search: Uses embeddings for intelligent document retrieval
💬 Conversational AI: Generates context-aware responses using Groq API
⚡ Fast & Efficient: Leverages ChromaDB for quick vector similarity search

🚀 Installation Prerequisites

Python 3.8 or higher
pip package manager

Setup Steps

Clone the repository:

bashgit clone https://github.com/mohamadlamg/Ready-tensor-RAG-assistant.git
cd Ready-tensor-RAG-assistant

Create a virtual environment (recommended):

bashpython -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate

Install dependencies:

bashpip install -r requirements.txt

Configure API keys:

Create a .env file at the project root:

envOPENAI_API_KEY=your_openai_key_here
GROQ_API_KEY=your_groq_key_here

💻 Usage

Basic Usage
bashpython app.py

Code Example

Here's how the RAG system works:

pythonfrom vectordb import VectorDB
from langchain_groq import ChatGroq

Initialize vector database

vector_db = VectorDB()

Load and process documents

vector_db.load_documents("data/")

Query the system

query = "What is metaprogramming in Python?"
response = vector_db.query(query)
print(response)

Adding Your Own Documents
Simply place your .txt files in the data/ folder:
data/
├── document1.txt
├── document2.txt
└── your_document.txt

📁 Project Structure
Ready-tensor-RAG-assistant/
│
├── app.py # Main application
├── vectordb.py # Vector database management
├── requirements.txt # Python dependencies
├── .env # API keys (not tracked by git)
├── .gitignore # Git ignore rules
│
└── data/ # Source documents
├── Building_Modern_GUIs_with_Tkinter_and_Python.txt
└── Metaprogramming-with-Python.txt

🛠️ Technologies Used

TechnologyPurposeLangChainFramework for LLM applicationsChromaDBVector database for embeddingsGroqFast LLM inferenceHuggingFaceText embeddings
🔧 Configuration

Model Selection

You can customize the LLM model in app.py.

📊 How It Works
mermaidgraph LR
A[Documents] --> B[Text Splitter]
B --> C[Embeddings]
C --> D[Vector DB]
E[User Query] --> F[Retrieval]
D --> F
F --> G[LLM]
G --> H[Response]

Document Loading: Load text files from the data/ folder

Text Splitting: Break documents into manageable chunks

Embedding: Convert text chunks into vector representations

Storage: Store vectors in ChromaDB

Query: User asks a question

Retrieval: Find relevant document chunks

Generation: LLM generates answer based on context

Thanks to the LangChain community for the amazing framework
HuggingFace for providing open-source embeddings
Groq for fast LLM inference

📧 Contact
Project Link: https://github.com/mohamadlamg/Ready-tensor-RAG-assistant