🧠 Local RAG Chatbot PoC with Java and Spring Boot

I've developed a Proof of Concept (PoC) for a Retrieval-Augmented Generation (RAG) chatbot using Java 21 and Spring Boot.
The goal was to experiment with building a fully offline, local-first AI system that combines web scraping, embeddings, and a lightweight LLM.

The application integrates a local LLM (Mistral via Ollama), an in-memory vector store, and the all-MiniLM-L6-v2 embedding model (via DJL).
You can submit URLs, extract their content via Firecrawl, generate embeddings from that content, and then query the system for context-aware responses — all running 100% locally.

🔗 You can find the full source code on GitHub: https://github.com/lucadallavecchia/rag-poc

---

🎯 Goal

The idea was to explore how a Spring Boot application could:

• 🔍 Scrape HTML content from a web page
• ✂️ Chunk and embed the content using a local embedding model
• 🧠 Use a local LLM (Mistral via Ollama) to respond to user questions
• 💬 Expose all of this via a REST API

Everything runs offline and without depending on external LLM APIs or SaaS tools.

---

🛠️ Tech Stack

---

🔧 How it works

1. The user sends a URL to the /documents endpoint
2. The page is scraped using Firecrawl
3. The HTML content is cleaned, chunked and embedded
4. Embeddings are stored in memory
5. When the user sends a query to /ask, the top relevant chunks are selected using cosine similarity
6. A prompt (Context + Question) is sent to the local Mistral model via Ollama
7. The chatbot replies with a context-aware answer — all offline

---

▶️ Quick Start

# Prerequisites
- Java 21
- Ollama with the `mistral` model installed
- Firecrawl API key

# Clone and run the project
git clone https://github.com/lucadallavecchia/rag-poc
cd rag-poc
./gradlew bootRun

![RAC PoC Schema.png](RAC%20PoC%20Schema.png)