RAG-based AI Assistant with LlamaIndex, ChromaDB, CutomMemory and UI
Table of contents
From scratch: I just wrapped a major milestone on my RAG System, a hands-on build I developed entirely from scratch to deeply understand how modern Retrieval-Augmented Generation systems really work under the hood.
What’s inside?
- Multi-provider LLM integration (Groq, OpenAI, Gemini, DeepSeek) unified through a custom orchestration layer
- Secure, encrypted API key handling with Fernet and user-specific key derivation
- Intelligent memory management: token-aware state, automatic summarization, and context-trim logic
- Vector search powered by ChromaDB + HuggingFace BGE embeddings for precise, filtered retrieval
- Robust document pipeline: structured PDF ingestion, multi-format parsing, chunking, metadata enrichment
- Strict system prompting to guarantee retrieval fidelity and prevent hallucinations
UI Features:
- API key configuration modal (no command-line hassle, secure input right in the UI)
- Document upload interface with drag-and-drop support
- One-click database clearing and chat history reset
- Copy-to-clipboard for individual messages
- Export full conversation as PDF
- Real-time chat with typing indicators and message timestamps
What I learned:
- Designing token-aware conversational workflows and summarization transitions
- Building secure local encryption flows that balance safety with usability
- Architecting modular RAG flows: document prep → embedding store → retrieval → contextual LLM prompting → continuity
- Navigating trade-offs across different LLM ecosystems and provider abstractions
- Applying effective UX patterns for AI assistants, especially around clarity and responsiveness
Notes & next steps:
As a solo learning project, a few rough edges are expected. Streamlit was perfect for rapid iteration, but a production framework will be my next step. Plans include user isolation, persistent chats, cloud vector backends, streaming responses, and analytics.
Open to all:
Feel free to explore it, learn from it, fork it, or extend it.
If you’re working on RAG systems, or curious about the architectural side of AI products — I’m always open to connect.
More details are in the repo (README.md) and the architectural deep-dive (DOCUMENTATION.md).
