Abstract

Large Language Model (LLM) applications face security risks on both sides of the inference boundary. Malicious user prompts can attempt to override system instructions, while model outputs may disclose personally identifiable information (PII), credentials, or proprietary intellectual property. Traditional Data Loss Prevention (DLP) tools often rely on static keyword lists and cannot detect semantically paraphrased code leaks or adversarial jailbreak phrasing framed in administrative language.

Sentinel-LLM V3 extends the V2 bi-directional guardrail with a persistent ChromaDB Forbidden Vault and a built-in Vault Management UI, enabling security teams to scale proprietary IP protection beyond demo-scale hardcoded snippets. The system combines four specialized detection engines – prompt injection classification, NER-based PII analysis, regex secret matching, and hybrid semantic IP comparison – within a unified orchestration layer that supports blocking, redaction, and cross-layer hardening. When inbound analysis flags injection or low-confidence safe classifications, outbound intellectual property (IP) detection automatically becomes more sensitive, reducing the attack surface for chained exploits.

This document describes the problem context, system architecture, detection methodology, vault persistence layer, decision logic, implementation details, and reproducibility instructions for Sentinel-LLM V3. It is intended for publication on Ready Tensor and for use by security engineers, ML practitioners, and researchers evaluating LLM guardrail patterns.

Keywords: LLM security, prompt injection, data leakage, guardrails, Presidio, DeBERTa, ChromaDB, sentence-transformers, vector database, OWASP LLM Top 10, bi-directional filtering

---

Table of Contents

1. Introduction
2. Threat Model and Design Goals
3. System Architecture
4. Detection Engines
5. Forbidden Vault and Vector Persistence
6. Cross-Layer Hardening and Decision Logic
7. Implementation Reference
8. Configuration and Thresholds
9. Reproducibility
10. Evaluation Scenarios
11. Limitations
12. Future Work
13. References

---

1. Introduction

1.1 Problem Statement

Deploying LLMs in production introduces two distinct but related security challenges:

1. Inbound threats (OWASP LLM01 – Prompt Injection): Attackers craft inputs designed to bypass system instructions, extract hidden prompts, or coerce the model into unsafe behavior. These attacks may use explicit jailbreak language or subtle administrative phrasing that evades naive filters.

2. Outbound threats (OWASP LLM06 – Sensitive Information Disclosure): Model outputs may contain PII, API keys, database connection strings, or fragments of proprietary source code either from training data memorization, context window leakage, or hallucination.

Most early LLM guardrails focus exclusively on output filtering. Sentinel-LLM V3 evaluates both user prompts and model outputs through a coordinated pipeline, enabling defense-in-depth without requiring access to model weights or internal logits.

1.2 Version History and V3 Enhancements

Sentinel-LLM V1 provided outbound scanning for PII, secrets, and semantic code similarity. V2 added inbound prompt injection detection, cross-layer hardening, and hybrid IP comparison against an in-memory vault. V3 replaces the demo-scale vault with enterprise-ready persistence and management:

1.3 Intended Use Cases

• Security engineering portfolios demonstrating practical LLM guardrail design
• Pre-production evaluations of LLM application risk before middleware integration
• Educational demonstrations of OWASP LLM Top 10 mitigations
• Enterprise IP protection workflows where proprietary snippets are managed without redeploying code
• Foundation for production middleware (planned FastAPI wrapper)

---

2. Threat Model and Design Goals

2.1 Assets Protected

2.2 Assumptions

• Text is evaluated before delivery to the end user (post-generation filter) or before forwarding to the LLM (pre-inference filter).
• The guardrail operates as an external sidecar; it does not modify model weights.
• Proprietary code snippets are stored in a local ChromaDB collection at ./security_vault/ and managed via the Streamlit UI or programmatic API.
• Models are downloaded from Hugging Face on first run and cached locally.
• ChromaDB uses the same embedding model (all-MiniLM-L6-v2) as the detector for consistent similarity scoring.

2.3 Out of Scope

• Model-level adversarial fine-tuning or RLHF alignment
• Network-level DLP for non-LLM traffic
• Encrypted or multimodal inputs (images, audio)
• Real-time streaming token-by-token filtering
• Multi-tenant vault isolation or access control (single local deployment)

2.4 Design Goals

1. Hybrid detection: Combine ML classification with deterministic heuristics to reduce single-point-of-failure risk.
2. Modularity: Isolate each engine in dedicated modules for independent testing and replacement.
3. Scalable vault: Persist protected IP in a vector database rather than hardcoded in source code.
4. Operational usability: Provide a UI for vault CRUD operations without code redeployment.
5. Tunability: Expose confidence thresholds for security teams to balance false positives and false negatives.
6. Explainability: Return structured findings with labels, scores, and detection methods.

---

3. System Architecture

3.1 Component Overview

Sentinel-LLM V3 consists of four Python modules and a two-tab Streamlit presentation layer:

┌──────────────────────────────────────────────────────────────────────────┐
│                         app.py (Streamlit UI)                            │
│   Tab 1: Security Audit — sliders, metrics, sanitized output             │
│   Tab 2: Vault Management — add, browse, wipe Forbidden Vault            │
└───────────────────────────────┬──────────────────────────────────────────┘
                                │
                                ▼
┌──────────────────────────────────────────────────────────────────────────┐
│                    detector.py (LLMLeakDetector)                         │
│                         run_report() orchestrator                        │
│                                                                          │
│  ┌─────────────────┐   ┌──────────────┐   ┌─────────────────────┐     │
│  │ Injection       │   │ Presidio     │   │ Secret Matcher      │     │
│  │ Detector        │   │ PII Scanner  │   │ (Regex)             │     │
│  │ (inbound)       │   │ (outbound)   │   │ (outbound)          │     │
│  └────────┬────────┘   └──────┬───────┘   └──────────┬──────────┘     │
│           │                 │                       │                 │
│           │    Cross-Layer  │                       │                 │
│           └────── Hardening ──┼───────────────────────┘                 │
│                             ▼                                            │
│                  ┌─────────────────────┐                                 │
│                  │ Hybrid IP Comparator │                                │
│                  │ (outbound)           │                                │
│                  └──────────┬──────────┘                                 │
│                             │                                            │
│                             ▼                                            │
│                  ┌─────────────────────┐                                 │
│                  │ Decision Logic       │                                │
│                  │ BLOCK or CLEAN       │                                │
│                  └─────────────────────┘                                 │
└───────────────────────────────┬──────────────────────────────────────────┘
                                │
              ┌─────────────────┴─────────────────┐
              ▼                                   ▼
┌─────────────────────────────┐   ┌─────────────────────────────────────┐
│ injection_engine.py         │   │ vault_manager.py (VaultManager)     │
│ InjectionDetector           │   │ ChromaDB PersistentClient           │
│ DeBERTa-v3 + heuristics     │   │ forbidden_vault collection          │
└─────────────────────────────┘   └─────────────────────────────────────┘

3.2 Processing Pipeline

Every security audit follows a fixed four-stage pipeline:

Input Text
    │
    ▼
[Stage 1] Inbound Injection Scan
    │
    ▼
[Stage 2] Cross-Layer Hardening (adjust IP threshold if suspicious)
    │
    ▼
[Stage 3] Outbound Scans (PII, Secrets, IP Leakage via ChromaDB)
    │
    ▼
[Stage 4] Decision Logic (BLOCK with reason, or CLEAN with redaction)
    │
    ▼
Structured Report (status, final_text, findings)

3.3 Technology Stack

---

4. Detection Engines

4.1 Inbound: Prompt Injection Detector

Module: injection_engine.py
Model: protectai/deberta-v3-base-prompt-injection
Approach: Hybrid ML classification with regex override

The injection detector implements a three-step scan (unchanged from V2):

Step A – Heuristic Signature Matching

Before invoking the ML model, the engine checks input against a curated list of high-risk regex patterns:

If any pattern matches, the engine returns INJECTION (Heuristic) with confidence 1.0, bypassing the ML classifier entirely.

Step B – ML Behavioral Classification

When no heuristic match occurs, text is passed to the DeBERTa-v3 classifier via Hugging Face's pipeline("text-classification"). The model returns a label (SAFE or injection-related) and a confidence score.

Step C – Low-Confidence Safe Flagging

If the model returns SAFE with confidence below 90%, the input is relabeled as SUSPICIOUS. This conservative policy triggers cross-layer hardening without immediately blocking the request.

Return values: (label: str, score: float)

---

4.2 Outbound: PII Detection

Module: detector.py → scan_pii()
Engine: Microsoft Presidio AnalyzerEngine

Presidio performs Named Entity Recognition (NER) over English text, detecting:

• EMAIL_ADDRESS
• PERSON
• US_SSN (via custom pattern recognizer)

A custom SSN recognizer supplements Presidio's default registry:

ssn_pattern = Pattern(
    name="ssn_pattern",
    regex=r"\b\d{3}-\d{2}-\d{4}\b",
    score=1.0
)

Results are filtered by a configurable confidence threshold (pii_threshold, default 0.4). When the final status is CLEAN, detected PII is replaced via Presidio's AnonymizerEngine with the token [REDACTED_PII].

---

4.3 Outbound: Secret Scanning

Module: detector.py → scan_secrets()
Approach: Deterministic regex matching

Two secret categories are defined:

Secrets are redacted with [REDACTED_SECRET] during the clean-path output sanitization.

---

4.4 Outbound: Hybrid IP Leakage Detection

Module: detector.py → scan_code_leakage()
Vault backend: vault_manager.py → VaultManager
Approach: Two-layer hybrid (keyword first, ChromaDB semantic fallback)

Unlike V2's in-memory vault with pre-computed embeddings, V3 queries a persistent ChromaDB collection. The engine uses a two-pass strategy:

Layer A – Keyword Match (Database Documents)

All documents in the Forbidden Vault are retrieved and scanned for identifier overlap:

clean_name = snippet.split('(')[0].replace('def ', '').replace('=', '').strip()
if clean_name.lower() in text_lower and len(clean_name) > 5:
    # Match recorded with score 1.0, method "Keyword Match (DB)"

This extracts the primary identifier (e.g., function or variable name) from each stored snippet. If that identifier appears in the input text (case-insensitive) and exceeds five characters, a match is recorded immediately.

Layer B – ChromaDB Semantic Search

If Layer A finds no matches, the input is queried against the vault:

results = self.vault.query_vault(text, n_results=1)
distance = results['distances'][0][0]
similarity = 1 - distance  # Cosine distance → similarity

ChromaDB is configured with hnsw:space: cosine. Matches exceeding the configurable code_threshold (default 0.7) are flagged with method ChromaDB Semantic Search.

This hybrid design catches both explicit identifier mentions and paraphrased reproductions that keyword lists alone would miss, while scaling to arbitrarily large vault corpora.

---

5. Forbidden Vault and Vector Persistence

5.1 VaultManager Overview

Module: vault_manager.py
Class: VaultManager
Storage path: ./security_vault/ (gitignored)

The vault manager wraps ChromaDB's persistent client and exposes four operations:

5.2 Collection Configuration

self.client = chromadb.PersistentClient(path=db_path)
self.emb_fn = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name="all-MiniLM-L6-v2"
)
self.collection = self.client.get_or_create_collection(
    name="forbidden_vault",
    embedding_function=self.emb_fn,
    metadata={"hnsw:space": "cosine"}
)

Each snippet is stored with a UUID identifier and default metadata {"type": "proprietary_code"}. Custom metadata can be passed on insert for future filtering extensions.

5.3 Vault Management UI

The Vault Management tab in app.py provides operational controls:

1. Add snippet: Text area + form submit → calls detector.vault.add_to_vault()
2. Browse snippets: Expandable list of all documents in the collection
3. Wipe database: Button → calls detector.vault.clear_vault()

The vault starts empty on first run. Security teams populate it with organization-specific code before running IP leak evaluations.

5.4 Embedding Consistency

Both ChromaDB indexing and query-time embedding use all-MiniLM-L6-v2 via ChromaDB's SentenceTransformerEmbeddingFunction. This ensures that documents added through the UI are embedded with the same model used during semantic search, avoiding score drift between ingestion and query paths.

---

6. Cross-Layer Hardening and Decision Logic

6.1 Cross-Layer Hardening

When the injection scan returns any label other than SAFE (including SUSPICIOUS), the IP similarity threshold is automatically reduced:

effective_code_threshold = max(0.1, code_threshold - 0.2)

Rationale: Adversarial prompts that attempt to extract proprietary logic often combine injection techniques with indirect code requests. Tightening outbound IP detection during suspicious inbound states reduces the window for chained attacks.

6.2 Blocking Conditions

The response is BLOCKED when either condition is true:

1. Injection label starts with INJECTION (includes heuristic detections)
2. One or more IP leakage findings are present

Blocked responses return a standardized message:

[BLOCKING RESPONSE: {reason} DETECTED]

where {reason} is the injection label or IP_LEAK.

6.3 Clean Path and Redaction

When no blocking condition is met, status is CLEAN. The pipeline applies sequential redaction:

1. Presidio anonymizes PII entities → [REDACTED_PII]
2. Secret values are string-replaced → [REDACTED_SECRET]

PII and secrets do not trigger blocking; they are sanitized. Blocking is reserved for injection and IP leakage.

6.4 Report Schema

The run_report() method returns a structured dictionary:

{
    "status": "BLOCKED" | "CLEAN",
    "final_text": str,          # Redacted output or block message
    "findings": {
        "inj": (label: str, score: float),
        "pii": list,              # Presidio RecognizerResult objects
        "secrets": list,          # {"type": str, "value": str}
        "leaks": list             # {"score": float, "matched_snippet": str, "method": str}
    }
}

Detection methods in leaks are either Keyword Match (DB) or ChromaDB Semantic Search.

---

7. Implementation Reference

7.1 Module Responsibilities

7.2 Core Orchestration

def run_report(self, text: str, pii_threshold: float, code_threshold: float):
    inj_label, inj_score = self.scan_injection(text)

    is_suspicious = inj_label != "SAFE"
    effective_code_threshold = (
        code_threshold if not is_suspicious
        else max(0.1, code_threshold - 0.2)
    )

    pii = self.scan_pii(text, pii_threshold)
    secrets = self.scan_secrets(text)
    code_leaks = self.scan_code_leakage(text, effective_code_threshold)

    if inj_label.startswith("INJECTION") or code_leaks:
        status = "BLOCKED"
        reason = inj_label if inj_label.startswith("INJECTION") else "IP_LEAK"
        redacted = f"[BLOCKING RESPONSE: {reason} DETECTED]"
    else:
        status = "CLEAN"
        # Presidio anonymization + secret string replacement

    return {"status": status, "final_text": redacted, "findings": {...}}

7.3 Model Loading and Caching

• DeBERTa-v3 and full detector: Loaded once via @st.cache_resource in the Streamlit app, preventing reload on every interaction.
• ChromaDB embeddings: Managed by ChromaDB's embedding function; documents are embedded on insert and at query time.
• Presidio: AnalyzerEngine and AnonymizerEngine are instantiated once per detector lifecycle.

7.4 First-Run Behavior

On first execution:

1. Hugging Face downloads protectai/deberta-v3-base-prompt-injection (~440 MB)
2. Sentence-Transformer weights for all-MiniLM-L6-v2 (~90 MB) are fetched for ChromaDB embedding
3. ChromaDB creates ./security_vault/ with an empty forbidden_vault collection
4. SpaCy model en_core_web_lg is installed via requirements.txt

Expect 2–5 minutes for initial setup depending on network speed.

---

8. Configuration and Thresholds

8.1 UI Controls

The sidebar also displays vault and inbound engine status indicators (ChromaDB persistent, DeBERTa-v3).

8.2 Internal Constants

8.3 Vault Population

V3 does not ship with pre-populated vault entries. Recommended workflow:

1. Launch the application
2. Navigate to Vault Management
3. Add proprietary snippets (function signatures, connection strings, internal logic)
4. Run audits in Security Audit against test inputs

Example starter snippet:

def internal_secure_auth_protocol(user_id, secret_salt):

---

9. Reproducibility

9.1 Prerequisites

• Python 3.11 or later
• Virtual environment (recommended)
• Approximately 2 GB disk space for model artifacts
• Internet access for first-run model downloads

9.2 Installation

git clone https://github.com/MANU-de/llm-leak-detector.git
cd llm-leak-detector
python -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -r requirements.txt

9.3 Running the Application

streamlit run app.py

The application opens in the default browser at http://localhost:8501 with two tabs: Security Audit and Vault Management.

9.4 Key Dependencies

Note: The pinned requirements.txt reflects a full environment freeze, including CUDA-related packages from the author's development machine. CPU-only deployments may require dependency adjustment.

9.5 Repository Structure

llm-leak-detector/
├── app.py                 # Streamlit UI (Security Audit + Vault Management)
├── detector.py            # Main orchestrator and outbound engines
├── injection_engine.py    # Inbound injection detector
├── vault_manager.py       # ChromaDB Forbidden Vault operations
├── security_vault/        # Local ChromaDB storage (gitignored, created at runtime)
├── requirements.txt       # Pinned dependencies
├── README.md              # Project overview
├── LICENSE                # Apache 2.0
└── docs/
    ├── sentinel-llm-v2-technical-documentation.md
    └── sentinel-llm-v3-technical-documentation.md

---

10. Evaluation Scenarios

The following test inputs validate each detection layer. Use the Security Audit tab unless noted otherwise.

10.1 Prompt Injection (Expected: BLOCKED or SUSPICIOUS with hardening)

10.2 PII Leakage (Expected: CLEAN with redaction)

Input:

The user John Doe (j.doe@email.com) requested an update for SSN 000-11-2222.

Expected: Status CLEAN; names, email, and SSN replaced with [REDACTED_PII].

10.3 Secret Leakage (Expected: CLEAN with redaction)

Input:

To access the production DB, use API_KEY: sk-ant-api03-abcdefg12345.

Expected: Status CLEAN; API key replaced with [REDACTED_SECRET].

10.4 IP Leakage – Keyword Match (Expected: BLOCKED)

Setup (Vault Management tab): Add snippet:

def internal_secure_auth_protocol(user_id, secret_salt):

Input (Security Audit tab):

I will create a function called internal_secure_auth_protocol(user_id, secret_salt) for the backend.

Expected: Status BLOCKED; method Keyword Match (DB).

10.5 IP Leakage – Semantic Match (Expected: BLOCKED at lower thresholds)

Setup: Add a longer proprietary function to the vault.

Input: Paraphrased description of vault logic without exact identifier names.

Expected: Status BLOCKED when ChromaDB similarity exceeds threshold; method ChromaDB Semantic Search.

---

11. Limitations

1. Single-tenant local vault: ChromaDB runs as a local persistent store with no built-in authentication or multi-tenant isolation.
2. Empty default vault: IP leak detection requires manual vault population before meaningful IP tests can run.
3. English-only: Presidio and SpaCy are configured for English (language='en'). Multilingual PII detection is not supported.
4. Batch, not streaming: The guardrail evaluates complete text blocks; token-by-token streaming protection is not implemented.
5. No audit export: Findings are displayed in the UI but not persisted to JSON/SIEM (planned).
6. Environment-specific dependencies: Full pip freeze output may include CUDA packages incompatible with all hardware.
7. Heuristic pattern coverage: Regex signatures cover known jailbreak families but cannot guarantee coverage of novel attack vectors.
8. PII does not block: Policy choice redacts PII rather than blocking; organizations requiring hard stops on any PII exposure may need policy adjustments.
9. Keyword scan loads all documents: Layer A iterates over every vault document on each audit; very large vaults may benefit from indexed keyword lookup in future versions.

---

12. Future Work

---

13. References

1. OWASP Foundation. OWASP Top 10 for Large Language Model Applications. https://owasp.org/www-project-top-10-for-large-language-model-applications/
2. Protect AI. deberta-v3-base-prompt-injection. Hugging Face Model Hub. https://huggingface.co/protectai/deberta-v3-base-prompt-injection
3. Microsoft. Presidio: Data Protection and De-identification SDK. https://microsoft.github.io/presidio/
4. Reimers, N., & Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. https://www.sbert.net/
5. Chroma. ChromaDB Documentation. https://docs.trychroma.com/
6. Project Repository. Sentinel-LLM / llm-leak-detector. https://github.com/MANU-de/llm-leak-detector
7. Technical Demo. Sentinel-LLM Loom Walkthrough. https://www.loom.com/share/848661f6acec4384bc4ad4a8ac797860

---