Insurance RAG Assistant - Project Documentation

Abstract

This project presents the development and implementation of an intelligent Insurance Retrieval-Augmented Generation (RAG) system that provides instant, accurate answers to insurance policy queries through advanced semantic analysis and multi-document processing capabilities.

The Insurance RAG Assistant addresses critical challenges in insurance information retrieval by combining semantic query optimization with multi-document RAG architecture. The system automatically analyzes query complexity and intent, dynamically adjusting search parameters without manual tuning. Key innovations include intelligent K-value optimization that adapts retrieval depth based on query patterns, plan-specific filtering for targeted searches, and a hybrid architecture supporting both REST API integration and real-time chat interfaces.

The system demonstrates significant improvements in query processing efficiency and accuracy through automatic semantic pattern detection, achieving optimal document retrieval across four insurance plan types (Activ Assure, Activ Secure, Activ Health, Activ One). The implementation leverages Azure OpenAI services with FAISS vector indexing and Django REST framework, providing a scalable solution for insurance document processing and query resolution.

Key Contributions:

• Semantic query analysis with automatic complexity assessment
• Dynamic K-value optimization eliminating manual parameter tuning
• Multi-plan comparison capabilities with source citation tracking
• Professional API design supporting complex query payloads
• Real-time chat interface with plan-specific filtering

Methodology

1. System Architecture and Design

1.1 Overall Architecture
The system implements a three-tier architecture with intelligent query processing:

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Streamlit     │    │   Django REST    │    │   Document      │
│   Frontend      │───▶│   API Backend    │───▶│   Processing    │
│                 │    │                  │    │                 │
│ - Chat Interface│    │ - Query Analysis │    │ - PDF Ingestion │
│ - Plan Selection│    │ - Semantic K Opt │    │ - Chunking      │
│ - Results Display│   │ - RAG Pipeline   │    │ - FAISS Index   │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌──────────────────┐
                       │ Semantic Query   │
                       │ Analyzer         │
                       │                  │
                       │ - Pattern Detect │
                       │ - K Optimization │
                       │ - Complexity     │
                       │   Assessment     │
                       └──────────────────┘

1.2 Core Components

• Backend (Django REST API)

  • views.py: REST API endpoints and request handling
  • semantic_query_analyzer.py: Intelligent query pattern detection
  • ingestion.py: PDF processing and vector index creation
  • utils.py: Shared utilities and helper functions
  • prompt_config.py: LLM prompt templates and configurations
  • logging_config.py: Comprehensive logging system

• Frontend (Streamlit)

  • insurance_chat.py: Interactive chat interface with plan selection

2. Data Processing Pipeline

2.1 Document Ingestion Process

# Document processing workflow
PDF Files → Text Extraction → Chunking → Embedding Generation → FAISS Index

2.2 Vector Index Creation

• Input: Insurance policy PDF documents (Activ Assure, Activ Secure, Activ Health, Activ One)
• Processing: Chunking with optimized sizes for insurance content
• Storage: FAISS vector database for efficient similarity search
• Metadata: Plan names, page numbers, and content previews for citation

2.3 Chunking Strategy

• Optimized chunk sizes for different insurance content types
• Overlap handling to maintain context continuity
• Metadata preservation for source tracking

3. Semantic Query Analysis Implementation

3.1 Query Pattern Detection
The system implements embedding-based similarity analysis to classify queries into patterns:

• Procedural Information: Claims processes, documentation requirements
• Benefit Queries: Coverage details, specific benefits
• Comparison Queries: Multi-plan comparisons
• Exclusion Queries: Coverage limitations and exclusions

3.2 Dynamic K-Value Optimization

# Automatic optimization logic
Query Input → Pattern Detection → Complexity Assessment → K-Value Calculation

Algorithm Features:

• Confidence scoring for pattern detection
• Automatic adjustment based on query complexity
• Fallback mechanisms for ambiguous queries

4. RAG Pipeline Implementation

4.1 Retrieval Process

• Semantic Search: FAISS-based vector similarity matching
• Plan Filtering: Optional filtering by specific insurance plans
• Dynamic K-Selection: Automatic optimization of document retrieval count
• Source Tracking: Page-level citation with content previews

4.2 Generation Process

• Context Injection: Retrieved documents formatted as context
• Prompt Engineering: Insurance-domain specific prompts
• Response Formatting: Structured answers with source citations
• Plan-Specific Information: Targeted responses based on selected plans

5. Technology Stack

5.1 Core Technologies

• Backend Framework: Django REST Framework
• Frontend: Streamlit for interactive chat interface
• Vector Database: FAISS for efficient similarity search
• LLM Service: Azure OpenAI (GPT-4 and text-embedding-ada-002)
• Document Processing: Python PDF processing libraries

5.2 Configuration Management

AZURE_OPENAI_API_KEY=your_api_key_here
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_VERSION=2024-02-15-preview
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=gpt-4
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=text-embedding-ada-002

6. API Design and Implementation

6.1 REST API Endpoints

• POST /api/build-index/: Vector index creation from PDF documents
• POST /api/query/: Enhanced query processing with semantic analysis
• GET /api/plan_names/: Available insurance plan retrieval
• GET /api/test-connection/: System health check

6.2 Enhanced Query Endpoint

POST /api/query/
{
  "q": "What documents are required for a cashless claim?",
  "k": "4",
  "plan_name": "Activ Assure,Activ Secure,Activ Health,Activ One"
}

Results

1. System Performance Metrics

1.1 Query Processing Capabilities

• Supported Insurance Plans: 4 comprehensive plans (Activ series)
• Document Types: PDF policy documents with full text processing
• Response Time: Real-time query processing through REST API
• Concurrent Users: Scalable architecture supporting multiple simultaneous users

1.2 Semantic Analysis Performance

"k_optimization": {
    "user_requested": 4,
    "system_calculated": 7,
    "pattern_detected": "procedural_info",
    "confidence": 0.8142491852568631,
    "reasoning": "Detected 'procedural_info' query pattern"
}

Query Pattern Recognition:

• Procedural Information: 81.4% confidence detection
• Benefit Queries: High accuracy in coverage identification
• Comparison Queries: Effective multi-plan analysis
• Automatic K-Value Optimization: Dynamic adjustment from user input to system-calculated optimal values

2. Functional Capabilities

2.1 Multi-Document RAG Performance

• Source Citation: Page-level references with content previews
• Cross-Plan Search: Simultaneous search across multiple insurance plans
• Content Filtering: Plan-specific result filtering and organization
• Comprehensive Answers: Detailed responses with structured formatting

2.2 Example Query Results

{
    "answer": "For a cashless claim under the Activ Secure insurance plan...",
    "source_docs": [
        {
            "plan_name": "Activ Health",
            "source_file": "Activ Health.pdf",
            "page_number": 35,
            "content_preview": "(iv) Payment receipt of any change in the travel booking..."
        }
    ],
    "total_docs_found": 7,
    "plans_searched": ["Activ Assure", "Activ Secure", "Activ Health", "Activ One"]
}

3. Technical Implementation Results

3.1 Architecture Benefits

• Modular Design: Clear separation between frontend, backend, and processing components
• Scalable Processing: FAISS vector indexing for efficient large-scale document retrieval
• Professional API: RESTful design supporting complex query payloads
• User Experience: Intuitive Streamlit interface with real-time chat capabilities

3.2 Integration Capabilities

• API-First Design: Professional endpoints for system integration
• Environment Configuration: Secure credential management
• Development Workflow: Streamlined setup and deployment process
• Monitoring: Comprehensive logging and error tracking

4. Business Impact and Applications

4.1 Use Case Coverage

• Customer Service: Instant policy information retrieval
• Claims Processing: Documentation requirement guidance
• Plan Comparison: Multi-product analysis capabilities

4.2 System Advantages

• No Manual Tuning: Automatic optimization eliminates parameter management
• Comprehensive Coverage: Supports complex multi-plan queries
• Professional Integration: API-ready for enterprise deployment
• User-Friendly Interface: Accessible through web-based chat interface

5. Limitations

5.1 Technical Constraints

• Limited to PDF document format; cannot process other file types (Word, HTML, web pages)
• FAISS vector database operates in-memory, requiring index rebuilding on server restarts and limiting scalability
• Dependency on Azure OpenAI API introduces rate limits, cost scaling concerns, and potential latency issues
• Single-server Django architecture without load balancing capabilities

5.2 Domain and Data Limitations

• System trained specifically for insurance domain with semantic patterns optimized for four Activ insurance plans only
• Static knowledge base requiring manual updates when policy documents change
• No real-time integration with current policy updates or regulatory changes
• English-language support only, lacking multi-language capabilities

5.3 Functional Limitations

• Cannot perform numerical calculations (premium computations, benefit calculations)
• Limited multi-step logical reasoning for complex insurance scenarios
• No conversation memory between queries in the chat interface
• Potential hallucination risk where system may generate plausible but incorrect information

5.4 Response Quality Issues

• Retrieved documents may not always directly answer specific queries
• Chunking strategy may split important contextual information across document boundaries
• Response consistency may vary for similar questions due to semantic search variations

Conclusion

The Insurance RAG Assistant successfully demonstrates advanced semantic query processing capabilities in the insurance domain. The system's intelligent K-value optimization and multi-document RAG architecture provide significant improvements over traditional keyword-based search systems.

Key Achievements:

• ✅ Automatic query pattern detection with high confidence scores
• ✅ Dynamic search optimization eliminating manual parameter tuning
• ✅ Multi-plan comparison capabilities with comprehensive source citation
• ✅ Professional API design supporting enterprise integration
• ✅ Real-time chat interface for immediate user interaction

Technical Innovation:
The semantic query analyzer represents a significant advancement in RAG system design, automatically adapting retrieval parameters based on query complexity and intent. This eliminates a major pain point in traditional RAG implementations where users must manually tune search parameters.