JusticeGPT — IPC Legal Copilot

Tags: RAG, Agentic AI, LangGraph, LangChain, Legal NLP, Chroma, FAISS
License: MIT
Version: 0.1.0

Table of contents

• Executive summary
• Problem statement & motivation
• Data
• Methodology
• Experiments & results
• Limitations & responsible AI
• Reproducibility
• Conclusion & future work
• References
• Code / Datasets / Files

Executive summary

JusticeGPT — IPC Legal Copilot is a multi-agent Retrieval-Augmented Generation (RAG) assistant for the Indian Penal Code (IPC). The IPC corpus is split into two halves and indexed into two vector stores (Chroma + FAISS). A condense node converts follow-up questions (e.g., “Is it bailable?”) into standalone queries using a single shared memory, then two retrieval agents query their respective stores and a synthesizer merges their outputs into one answer. The project includes a Flask UI with a “Thinking…” spinner and an auto-build step that creates vector stores on first run and skips rebuilding thereafter.

Problem statement & motivation

Single-index chatbots often fail when users ask follow-ups that rely on conversation context. Legal queries frequently use pronouns (“it”, “that section”), so retrieval must resolve these references before searching. JusticeGPT demonstrates a practical pattern:

• Shared memory retains context across turns.
• A condense step rewrites follow-ups into standalone questions for retrieval.
• Dual stores (Chroma + FAISS) increase coverage and modularity; a synthesizer yields a single, clean, statute-grounded answer.

Use cases: quick legal reference for students/educators, legal researchers, and developers learning agentic RAG.

Data

• Corpus: The Indian Penal Code (1860) as a public PDF (mirrors handled in the loader).
• Extraction: text-only using pypdf.
• Chunking: RecursiveCharacterTextSplitter with chunk_size=1500, chunk_overlap=200.
• Split: even split of chunks into:
  • ipc_part1 → Chroma (persistent dir: chroma_db/ipc_part1)
  • ipc_part2 → FAISS (saved at faiss_db/ipc_part2)
• Build behavior: On first run, the app downloads the PDF to docs/ and builds both stores. Later runs detect existing stores and skip rebuild.
• Licensing/Attribution: Verify compliance with any government document terms in your jurisdiction.

Methodology

High-level architecture

%%{init: {'securityLevel': 'loose', 'flowchart': {'htmlLabels': true}} }%%
flowchart LR
  U[User]:::user --> C[[Condense = follow-up → standalone]]:::condense
  C --> A1[Agent 1<br/>Retriever = Chroma /ipc_part1]:::agent
  C --> A2[Agent 2 Retriever = FAISS /ipc_part2]:::agent
  A1 --> S[[Synthesizer LLM merge]]:::synth
  A2 --> S
  S --> R((Final Answer)):::answer

  C -. uses history .- M[(Shared Memory = ConversationBufferMemory)]:::memory
  A1 -. context .- M
  A2 -. context .- M

  classDef user fill:#E3F2FD,stroke:#1E88E5,stroke-width:1.5px,color:#0D47A1;
  classDef condense fill:#FFF3E0,stroke:#FB8C00,stroke-width:1.5px,color:#E65100;
  classDef agent fill:#E8F5E9,stroke:#43A047,stroke-width:1.5px,color:#1B5E20;
  classDef synth fill:#F3E5F5,stroke:#8E24AA,stroke-width:1.5px,color:#4A148C;
  classDef answer fill:#E0F7FA,stroke:#0097A7,stroke-width:1.5px,color:#006064;
  classDef memory fill:#FFFDE7,stroke:#FBC02D,stroke-width:1.5px,color:#7F6F00;

  linkStyle 0 stroke:#1E88E5,stroke-width:1.5px;
  linkStyle 1 stroke:#43A047,stroke-width:1.2px;
  linkStyle 2 stroke:#43A047,stroke-width:1.2px;
  linkStyle 3 stroke:#8E24AA,stroke-width:1.2px;
  linkStyle 4 stroke:#8E24AA,stroke-width:1.2px;
  linkStyle 5 stroke:#0097A7,stroke-width:1.5px;

Flow:
User → Condense (rewrite follow-up → standalone) → Agent 1 (Chroma, part-1) + Agent 2 (FAISS, part-2) → Synthesizer (LLM merge) → Final answer
Shared memory (one ConversationBufferMemory) is used by condense and both agents to keep context.

Components

• Condense node: Uses chat history + --CONDENSE-- prompt (prompts/promt.txt) to rewrite the latest message into a standalone query.
• Agent 1 (Chroma): RetrievalQA(stuff) over ipc_part1, top-k=4.
• Agent 2 (FAISS): RetrievalQA(stuff) over ipc_part2, top-k=4.
• Synthesizer: Merges Agent-1/Agent-2 drafts using --SYNTH-- prompt to produce a concise, statute-grounded final.
• Graph: LangGraph 0.5 typed state with nodes {condense, agent1, agent2, synth}; state includes query, standalone_query, ans1, ans2, final.
• Models/Embeddings: ChatOpenAI (default gpt-4o, temperature=0), OpenAIEmbeddings (default text-embedding-3-small).

Vector store build (first-run pipeline)

%%{init: {'securityLevel': 'loose', 'flowchart': {'htmlLabels': true}} }%%
flowchart LR
  S{First Run?}:::decision
  S -- No --> Load[Load Existing Stores<br/>Chroma + FAISS]:::load
  Load --> Ready((Ready to Serve)):::ready

  S -- Yes --> DL[Download IPC PDF]:::download --> EX[Extract Text]:::extract --> CH[Chunk & Split]:::chunk
  CH --> CBuild[Build Chroma = ipc_part1]:::chroma
  CH --> FBuild[Build FAISS = ipc_part2]:::faiss
  CBuild --> PS[Persist Indexes]:::persist
  FBuild --> PS
  PS --> Ready

  classDef decision fill:#FFFDE7,stroke:#FBC02D,stroke-width:1.5px,color:#6D4C00;
  classDef load fill:#DCEDC8,stroke:#7CB342,stroke-width:1.5px,color:#33691E;
  classDef ready fill:#E0F7FA,stroke:#00838F,stroke-width:1.5px,color:#005662;
  classDef download fill:#E3F2FD,stroke:#1E88E5,stroke-width:1.5px,color:#0D47A1;
  classDef extract fill:#F3E5F5,stroke:#8E24AA,stroke-width:1.5px,color:#4A148C;
  classDef chunk fill:#FFF3E0,stroke:#FB8C00,stroke-width:1.5px,color:#E65100;
  classDef chroma fill:#E8F5E9,stroke:#43A047,stroke-width:1.5px,color:#1B5E20;
  classDef faiss fill:#E8EAF6,stroke:#3949AB,stroke-width:1.5px,color:#1A237E;
  classDef persist fill:#ECEFF1,stroke:#607D8B,stroke-width:1.5px,color:#263238;

  linkStyle 0 stroke:#7CB342,stroke-width:1.2px;
  linkStyle 1 stroke:#00838F,stroke-width:1.5px;
  linkStyle 2 stroke:#1E88E5,stroke-width:1.2px;
  linkStyle 3 stroke:#8E24AA,stroke-width:1.2px;
  linkStyle 4 stroke:#FB8C00,stroke-width:1.2px;
  linkStyle 5 stroke:#43A047,stroke-width:1.2px;
  linkStyle 6 stroke:#3949AB,stroke-width:1.2px;
  linkStyle 7 stroke:#607D8B,stroke-width:1.2px;
  linkStyle 8 stroke:#00838F,stroke-width:1.5px;

Logic:
If stores exist → load & serve.
Else → download IPC PDF → extract → chunk → split → build Chroma & FAISS → persist → serve.

Experiments & results

This release emphasizes functionality and reproducibility. Suggested evaluation protocol:

• Qualitative QA set: 25–50 IPC questions (e.g., sections 302/304/420; “bailable?”, “compoundable?”, “punishment?”).
• Follow-up robustness: 2–3 pronoun-based follow-ups per question (“Is it bailable?”, “What’s the punishment for that?”).
• Human ratings: score Correctness, Groundedness, Clarity (1–5).
• Retrieval diagnostics: log the retrieved spans for spot checks.
• Latency: measure median per turn; first run includes one-time build.

Expected behavior: After warm-up, retrieval is local/fast; latency dominated by LLM calls. Condense significantly improves follow-up accuracy vs. naive chat.

Limitations & responsible AI

• Not legal advice; consult qualified professionals for decisions.
• IPC amendments may require refreshing the corpus/indexes.
• PDF extraction can contain artifacts (line breaks/OCR).
• FAISS loading uses allow_dangerous_deserialization=True for your own locally-saved index; never load untrusted pickles.
• Hallucinations are possible; consider adding a citations panel linking retrieved spans.

Reproducibility

Environment

Pinned in requirements.txt (notable pins: numpy<2.0, langgraph==0.5.4, langchain==0.2.*, langchain-openai, langchain-chroma, chromadb==0.4.*, faiss-cpu, pypdf, flask).

Setup

python3.13 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
cp .env.example .env
# Edit .env:
# OPENAI_API_KEY=sk-...
# (optional) OPENAI_CHAT_MODEL=gpt-4o
# (optional) OPENAI_EMBED_MODEL=text-embedding-3-small
# (optional) CHROMA_TELEMETRY=FALSE

You can watch the video here: