🧠 MemoryGPT – Abstract

MemoryGPT is a conversational AI system enhanced with long-term memory, enabling users to interact with their personal or organizational knowledge base in real time. By combining document parsing, semantic search, and natural language generation, MemoryGPT allows users to upload various document types (PDF, DOCX, TXT), store them as vector embeddings, and query them contextually.

This system leverages FAISS for vector storage and retrieval, LangChain for orchestration, and Cohere/OpenAI embeddings for semantic understanding. The user interface is built with React.js, delivering a rich, animated chat experience with visual memory trails, document metadata, and upload management.

The backend is powered by Flask, supporting modular APIs for chat, memory handling, and document ingestion. The system is cloud-deployable, optimized for Render (Backend) and Vercel (Frontend), supporting both local development and scalable cloud workflows.

Key Features:

• Upload, embed, and store documents for intelligent querying.
• Ask context-aware questions and get traceable answers.
• Visual memory timeline with session recall and clearing.
• Lightweight vector search powered by FAISS.
• Seamless deployment to Render and Vercel.

Use Cases:

• Internal knowledge base for teams
• Resume or contract analysis
• Research assistant for uploaded literature
• Document-heavy customer supports

MemoryGPT bridges the gap between static file storage and interactive knowledge retrieval, creating a memory-centric experience for users and organizations.

---

📘 Introduction

In today's information-rich environments, organizations and individuals handle vast amounts of unstructured data such as PDFs, Word documents, and text files. Accessing relevant insights from these documents efficiently and conversationally remains a significant challenge.

MemoryGPT is designed to solve this challenge by transforming how users interact with their document knowledge base. It enables users to upload files, converts them into vector embeddings using state-of-the-art embedding models, and stores them in a vector database (FAISS). Users can then query this knowledge conversationally, and the system intelligently retrieves the most relevant content using semantic search and provides human-like answers via an LLM.

MemoryGPT simulates long-term memory through persistent vector storage and session history tracking. Whether you’re building a personal assistant, research companion, or enterprise support agent, MemoryGPT can adapt to your needs.

The project is powered by:

• 🧠 LangChain: For orchestrating RAG pipelines
• 🔎 FAISS: For high-speed vector similarity search
• 🗂️ Cohere/OpenAI: For generating and embedding text
• ⚙️ Flask (Python): Backend for APIs and logic
• 💻 React (Vite + Tailwind): A beautiful, animated frontend
• 🌐 Render + Vercel: For deploying backend and frontend respectively

With an immersive UI, interactive chat interface, real-time uploads, and document-level memory tracking, MemoryGPT offers a powerful way to make your documents talk back.

---

---

🔍 Related Work

MemoryGPT builds on the ideas and progress of several related technologies and research efforts in the field of AI-powered document interaction, Retrieval-Augmented Generation (RAG), and conversational agents.

🧠 Retrieval-Augmented Generation (RAG)

RAG is a paradigm that combines document retrieval with powerful generative models to answer questions based on external context. Our architecture follows this approach by:

• Using vector embeddings for dense retrieval (FAISS)
• Feeding top-k results into a language model for contextual response generation

This method improves factuality and grounding, and is inspired by work from OpenAI, Meta AI, and the LangChain community.

📚 LangChain

LangChain is a framework for building LLM-based applications that involve external data sources. MemoryGPT uses LangChain’s:

• Document loaders
• Embedding and vector store integrations
• Prompting and chaining logic

It forms the backbone of our orchestration layer for document understanding.

📎 FAISS

Facebook AI Similarity Search (FAISS) is a library for efficient vector similarity search. It is widely used in both academic and industry settings for large-scale search applications. MemoryGPT uses FAISS to store and retrieve document embeddings efficiently.

🧾 Existing Projects

Projects such as:

• ChatGPT Retrieval Plugin
• Haystack by deepset
• LlamaIndex
• DocGPT

...have demonstrated the usefulness of combining LLMs with document stores. MemoryGPT draws from these tools but aims to offer a more user-friendly interface and persistent memory-like behavior tailored to both personal and enterprise knowledge workflows.

---

⚙️ Methodology

MemoryGPT is built on a Retrieval-Augmented Generation (RAG) pipeline that simulates long-term memory by persistently embedding and retrieving document knowledge. Below is a breakdown of the core components and how they work together.

---

1. 📤 Document Ingestion

• Users upload files (.pdf, .txt, .docx) via a React.js frontend.

• Files are sent to a Flask backend where they are parsed using libraries like:

  • PyMuPDF for PDFs
  • python-docx for DOCX
  • unstructured or html2text for text-based content

---

2. 🧠 Chunking & Embedding

• Parsed content is split into semantic chunks using LangChain’s recursive text splitter.

• Each chunk is embedded into a high-dimensional vector using:

  • Cohere Embeddings or OpenAI Embeddings (text-embedding-ada-002)

• Tags are extracted for each chunk using KeyBERT to support filtering and metadata enrichment.

---

3. 🗃️ Vector Storage (FAISS)

• All embeddings are stored in a FAISS index locally or on cloud (if needed).

• Each vector chunk contains metadata like:

  • Source filename
  • Page number
  • Tags
  • Timestamp

---

4. 💬 Query Handling

• Users enter natural language queries in the chat interface.

• The backend:

  • Embeds the query
  • Retrieves top-k relevant document chunks from FAISS
  • Passes the query + chunks to a LLM (e.g., Cohere Command or OpenAI GPT)

• The LLM synthesizes a final, human-readable response using prompt templates.

---

5. 🧠 Conversational Memory

• Previous chat history and responses are stored in a chat_memory.json file.
• A follow-up query can reference prior messages (like: “continue from last week”).
• MemoryTrail UI shows past interactions and context visually.

---

6. 🔁 Feedback Loop (Optional)

• Document chunks can be re-ranked or tagged post-chat.
• Future enhancements include user feedback-based re-weighting and relevance scoring.

---

This pipeline allows MemoryGPT to behave like a document-aware assistant with persistent, searchable memory. It not only responds intelligently but also remembers what matters.

---

Here’s a well-structured 🧪 Experiments section you can use in your README.md:

---

🧪 Experiments

To evaluate the performance and reliability of MemoryGPT, we conducted a series of practical experiments focusing on document understanding, retrieval accuracy, and conversational memory.

---

📁 1. Document Types Tested

All documents were chunked, embedded, and successfully stored in FAISS.

---

🔍 2. Retrieval Quality Test

We tested MemoryGPT with complex natural language queries and evaluated whether it could find the right content chunk and generate a meaningful answer.

⚠️ Partial answers occur when relevant chunks are split across multiple embeddings or when prompt length limitations truncate context.

---

🧠 3. Memory Trail Persistence

We evaluated the memory retention over multiple query sessions.

---

⚡ 4. Performance Benchmarks

Benchmarks were measured on a local machine with 16 GB RAM using FAISS (CPU-only). Cloud deployments may vary.

---

✅ Key Takeaways

• Works reliably with structured PDFs and Office documents.
• Retrieval performance is solid for keyword-rich queries.
• MemoryGPT performs well for short-term memory and is extensible to long-term memory using persistent storage.
• Query-context relevance may decrease when session memory is not properly refreshed or stored.

---

📊 Results

After deploying and testing MemoryGPT, the following results were observed across key metrics:

---

✅ Functional Outcomes

---

🧠 Memory Performance

---

🔬 Stress Test

---

🔁 Real-world Examples

• PowerBI.pdf: Summarization, KPI extraction, and comparisons worked as expected.
• Multiple files: Cross-document referencing was partially successful in dense contexts.

---

🏁 Conclusion

MemoryGPT successfully fulfills its role as a memory-augmented assistant capable of:

• Parsing and embedding documents
• Answering natural language questions grounded in source data
• Persisting memory between chats
• Offering a real-time chat UI experience

---

🧠 Discussion

MemoryGPT bridges the gap between static document search and interactive memory-driven conversation. By integrating document parsing, embedding, vector retrieval, and conversational memory, it brings a practical implementation of Retrieval-Augmented Generation (RAG) into real-world use.

---

🔍 Key Takeaways

1. Hybrid Architecture
   Combining FAISS for fast similarity search with generative models like Cohere/OpenAI enables MemoryGPT to both recall and reason. This hybrid RAG pipeline proves effective in knowledge-heavy applications.

2. Persistent Memory
   MemoryGPT simulates "organizational memory" by storing past conversations and file-based knowledge. Users can follow up on prior discussions, promoting continuity and long-term understanding.

3. User Experience Matters
   The immersive React-based frontend—featuring animated chat, real-time uploads, and file previews—greatly enhances usability. This shows how frontend design can elevate AI’s perceived intelligence.

4. Scalability Bottlenecks
   Deploying on limited platforms (like free-tier Render) introduced memory caps (512MiB), causing OOM errors during large file embeddings. Preloading via Git-based vector stores partially mitigates this, but long-term scalability requires dedicated infra (e.g. AWS, GCP, GPU-backed services).

---

🧪 Tradeoffs & Observations

---

🔄 Future Directions

• 🔁 Switch to database-based memory (e.g. PostgreSQL or Redis for long-term persistence)
• 🧱 Add hybrid retrieval (BM25 + dense for better recall in long text)
• 📡 Upgrade to streaming chat API for real-time generation and token-by-token response
• 🛡️ Enhance security (file upload sanitization, rate limiting, auth)
• ☁️ Deploy on more scalable cloud (AWS Lambda, Vercel Edge Functions + DB)

---

Let me know if you'd like to generate the next section (like 📦 Installation, 🚀 Deployment, or 🛠️ Architecture) or generate visuals for the README!

✅ Conclusion

MemoryGPT demonstrates the power of combining Retrieval-Augmented Generation (RAG), conversational memory, and user-friendly UI to build an intelligent assistant that remembers, reasons, and responds.

By blending structured file ingestion, vector-based semantic search (via FAISS), and generative reasoning (via Cohere/OpenAI), the project showcases how an AI system can act as a knowledgeable assistant—not just a chatbot. The persistent memory trail, immersive frontend, and document-aware responses give users a personalized and context-rich experience.

While the current setup focuses on local vector stores and lightweight hosting, the design is modular and scalable. With enhancements like database-backed memory, hybrid retrieval (BM25 + dense), and cloud-native deployment, MemoryGPT can evolve into a production-ready organizational memory system.

📚 References

1. LangChain Documentation
   https://docs.langchain.com
   → Used for chaining document loaders, embeddings, and retrieval in MemoryGPT.

2. FAISS: Facebook AI Similarity Search
   https://github.com/facebookresearch/faiss
   → Core vector store used for fast and efficient semantic search.

3. Cohere API Documentation
   https://docs.cohere.com
   → Used for both text embeddings and keyword extraction via the embed-english-light-v3.0 model and KeyBERT.

4. Render Deployment Guide
   https://render.com/docs/deploy-flask
   → Render platform used to deploy the Flask backend with web service configuration.

5. React Documentation
   https://reactjs.org/docs/getting-started.html
   → Used to build the interactive frontend chat interface.

6. Tiktoken (OpenAI Tokenizer)
   https://github.com/openai/tiktoken
   → Token-counting utility used to manage context window limits.

7. KeyBERT
   https://github.com/MaartenGr/KeyBERT
   → Used for lightweight keyword extraction from document chunks.

8. Waitress WSGI Server
   https://docs.pylonsproject.org/projects/waitress/en/stable/
   → Used to serve the Flask application in production (locally or on Render).

9. OpenAI API (optional)
   https://platform.openai.com/docs
   → Alternative or secondary LLM for answering document-based queries.

🙏 Acknowledgement

We would like to thank the following tools, platforms, and communities for enabling the development of MemoryGPT:

• OpenAI and Cohere for their powerful language models and embedding APIs.
• LangChain for simplifying the orchestration of document parsing, retrieval, and LLM chaining.
• FAISS by Facebook AI for making fast semantic search at scale possible.
• Render for easy and accessible cloud deployment of web services.
• React and the broader JavaScript ecosystem for enabling a dynamic and immersive frontend experience.
• KeyBERT for intuitive and lightweight keyword extraction.
• The open-source community for providing continuous innovation, support, and documentation.

Special thanks to mentors, teammates, and contributors who helped shape the direction, UI/UX, and backend architecture of this project.

📎 Appendix

🔧 Environment Variables

To run the backend, create a .env file in the backend/ directory with the following variables:

COHERE_API_KEY=your_cohere_api_key
EMBEDDING_MODEL=embed-english-light-v3.0

Optionally (if using OpenAI):

OPENAI_API_KEY=your_openai_key

📁 Project Structure

memorygpt/
├── backend/
│   ├── app.py
│   ├── api/
│   ├── services/
│   ├── utils/
│   ├── vector_store/
│   └── static/default_docs/
├── frontend/
│   ├── src/
│   │   ├── Pages/
│   │   ├── components/
│   └── public/
├── .env
├── requirements.txt
└── README.md

🧪 Sample Test Commands

# Start backend (locally)
cd backend
python app.py

# Start frontend
cd frontend
npm install
npm run dev

💻 Deployment Notes

• For Render: Bind to 0.0.0.0 and use PORT from the environment.
• Push prebuilt vector_store/ to Git if you want to avoid preloading every time.
• Avoid heavy preload jobs in production to stay under memory limits (e.g., 512MB on Render free tier).

---