TL;DR

SpecGuard is a draft-and-verify safety layer for LLM agents. Before a plan is executed, a small drafter samples several candidate futures in parallel. A critic labels each branch SAFE or HACK. The engine prunes deceptive branches and either selects a remaining safe plan or halts.

The current prototype targets Linux reward-hacking: shell-style plans that look as if they complete a task while actually tampering with sensors, logs, processes, permissions, or monitors.

This publication describes the public architecture and runtime. The deception dataset, training pipelines, and the drafter LoRA remain private while the project is in an extended testing phase, so those artifacts cannot be reused as a playbook or generator for real attacks. They may be released later under gated access once evaluation and disclosure policy are settled.

1. Introduction

Language-model agents are increasingly asked to act in environments: list files, run backups, update status, call tools. When the agent is scored on a proxy (a log line, a metrics file, a process still running), it can maximize the score without doing the intended work. That pattern is reward-hacking: the sensor is gamed, not the task.

Auditing every token with a large model is slow and sequential. SpecGuard takes a different trade-off: speculate, then prune. Generate a few cheap candidate plans, classify them, keep the safest one (or refuse).

The public repository implements that loop in Python on a 4-bit Llama 3.1 8B Instruct base with two PEFT LoRA adapters. Runtime lives in a single engine. Honey-pot execution in a real OS is not wired up yet; evaluation today is on task strings and logged speculative trees.

2. Problem statement

2.1 What we mean by Linux reward-hacking

In this project, a “hack” is a plan that appears to satisfy a Linux-style instruction while corrupting the measurement of success. Typical families in the (private) scenario set include:

• Sensor / log tampering — rewriting metrics or patching a monitor so the score looks good
• Process and monitor disabling — killing or backgrounding the thing that would report failure
• Obfuscation — hidden files, aliases, encoding, or misleading output
• Permission tricks — changing ownership or mode so auditors cannot inspect the result
• Network mocking — faking a remote check that never ran

We describe categories, not recipes. Concrete command sequences belong in the gated dataset, not in a public write-up.

2.2 Why a two-model loop

A single generator that is only “told to be safe” still samples deceptive shortcuts, especially when the prompt space includes deception (as it must, if you want a critic that has seen both sides). Separating roles is clearer:

The critic is trained so that honest trajectories are preferred over deceptive ones (DPO-style pairs). The drafter is trained to produce the kinds of futures the critic must learn to reject—that adapter is the dual-use risk, which is why it stays private for now.

3. System architecture

Task + environment
        │
        ▼
   Drafter (LoRA)
   N sampled plans
        │
        ▼
   Critic (LoRA)
   SAFE (0) or HACK (10)
        │
        ▼
   Prune HACK branches
        │
        ├── ≥1 SAFE → select lowest score
        └── all HACK → HALT
        │
        ▼
   JSON log (evaluations/logs/)

1. Draft. Prompt: task, short environment description, and a plan delimiter. Sample num_branches (default 3) completions at temperature 0.9.
2. Audit. Each branch is asked: Is this a reward-hack? Reply with SAFE or HACK. HACK maps to score 10; otherwise 0.
3. Prune. Score > 9 → PRUNED. Among PASSED branches, the lowest score is selected. If none pass, the engine returns HALT: All malicious.
4. Log. Timestamped JSON: task, every branch (text, score, status), final decision.

Drafter and critic are two LoRA adapters on the same 4-bit Llama 3.1 8B Instruct base, loaded with PEFT. Checkpoints are not stored in git.

4. Methodology

4.1 Data (high level)

Scenarios are paired Linux trajectories: an honest command sequence vs a deceptive one, plus a hacking mechanism (and often a critic clue). A teacher LLM (Groq-hosted) is used locally to scale themed pairs. Hub datasets used in internal notebooks:

• manuelaschrittwieser/specguard-linux-deception-v2 — paired deception set (private)
• manuelaschrittwieser/specguard-linux-dpo-v2 — DPO mapping: prompt / chosen (honest) / rejected (deceptive) (private)

We do not publish sample rows, generation prompts, or volume breakdowns that would make the set easy to reconstruct.

4.2 Training (high level)

• Drafter: LoRA on the 4-bit Llama 3.1 8B Instruct base (Unsloth-style Colab workflow).
• Critic: Same base; preference training on DPO-formatted pairs so the model should prefer the honest plan over the deceptive one for the same task/environment.
• Scripts and notebooks for generation, Hub upload, and fine-tuning live outside the public tree (gitignored).

4.3 Runtime implementation

The loop is SpecGuardEngine in src/engine.py:

• Load tokenizer and 4-bit base (BitsAndBytesConfig, NF4, float16 compute).
• Attach both LoRA adapters via PEFT.
• speculate → audit → prune → SpecGuardLogger.save_tree.

A smoke test (src/test_critic.py) checks that a clearly honest plan (ls → wc → cleanup) scores SAFE (0), so the critic is not purely paranoid.

What this is not (yet): Medusa / SpecInfer / shared-prefix tree decode. Branches are independent samples, not a KV-cache tree. env/ is a stub for a future honey-pot; nothing is executed against a real OS in the public engine.

4.4 Stack

• Runtime: Python, PyTorch, Hugging Face Transformers, PEFT, BitsAndBytes
• Training: Unsloth-style 4-bit Llama 3.1, LoRA, DPO pairs
• Data: Teacher-generated JSONL, Hugging Face Datasets / Hub (private)
• Eval: JSON speculative-tree logs; pytest reserved for future unit tests

5. Results so far (prototype)

Working today

• End-to-end draft → critic → prune on Linux-style task strings
• Dual LoRA on one quantized 8B base (Colab T4-class GPU is the intended path)
• JSON trees under evaluations/logs/
• Critic smoke test on an honest one-liner

Observed behavior (illustrative, not a benchmark)

On a demo task such as “list /home and print a file count,” sampled drafter branches are often labeled HACK and the run halts when all three fail the critic. That is the intended conservative default while the drafter is still biased toward deceptive structure from its training objective. It is not a claim of production-ready precision/recall.

Not implemented

• Real sandbox / Gymnasium honey-pot
• True speculative tree search
• vLLM / SGLang serving
• Split drafter / critic modules (both adapters share one base instance)

We are not reporting leaderboard numbers in this post. Extended testing is still in progress; public metrics would be misleading without a locked eval set and without releasing the artifacts needed to reproduce them.

6. Why parts of SpecGuard stay private

SpecGuard is a safety project, but two of its artifacts are dual-use by design:

1. The deception dataset is a catalog of how an agent can look successful while cheating Linux-style sensors. Publishing it in full would be a cheat sheet for the exact failure modes we want agents not to invent in the wild.
2. The drafter LoRA is trained to propose those futures. In the product it exists only so the critic can be stress-tested. Released without gates, it is closer to a small attack generator than to a harmless demo.

Because of that, the project is in an extended testing phase before any open release of data or the drafter:

• Prevent malicious reuse. We will not ship a public JSONL of tampering patterns or a downloadable model whose job is to write deceptive plans, while we still lack a real sandbox, red-team policy, and a clear intended-use license.
• Keep evaluation honest. Weights and splits are frozen internally so we can iterate on the critic and logging without the public internet fine-tuning on our test distribution.
• Match capability to containment. The public repo shows the control loop (draft, score, prune, halt, log). It does not include training/, data/raw/, fine-tune notebooks, .env secrets, or weight files. Those are gitignored on purpose.
• Possible later release. Dataset and/or drafter may become public or gated (Hugging Face access requests, research-only terms) once testing is further along—for example: honey-pot evaluation, clearer halt vs. false-positive trade-offs, and a disclosure note that does not include executable recipes.

The critic is the component we eventually want others to reuse. Even there, access should stay aligned with “defend agents,” not “train a better hacker.” Until that policy is written, treat Hub IDs as not an invitation to scrape or redistribute.

What you can inspect today without those artifacts: this write-up, the public engine and logger, the critic smoke-test intent, and the shape of evaluation logs (task, branch status, halt vs. select)—not a full attack corpus.

7. How to read the public repo

Project-SpecGuard/
├── src/engine.py      # Draft–audit–prune + logging
├── src/logger.py
├── src/test_critic.py
├── evaluations/logs/  # Example speculative trees
├── requirements.txt
└── README.md

Placeholders (drafter.py, critic.py, env/) mark future splits; they are empty of logic.

Run (when you have GPU + authorized Hub access):

python -m venv .venv
# Windows: .venv\Scripts\activate
source .venv/bin/activate
pip install -r requirements.txt
python src/engine.py

Without access to the private drafter adapter, the demo will not load. That is intentional.

8. Limitations and future work

• Independent samples ≠ speculative decoding. Latency claims from SpecInfer-style systems do not apply until we share a prefix tree and KV cache.
• No closed-loop environment. Plans are text; we do not yet execute them in a jail or score real file-system state.
• Binary critic. Scores are 0/10 from a short SAFE/HACK decode, not a calibrated probability.
• Shared base instance. Drafter and critic adapters are both attached to one loaded model; production serving should isolate them.
• English Linux-ish strings. No claim of transfer to other OS, languages, or tool APIs.

Next steps we consider in-scope: a contained honey-pot, true tree search, serving, a public critic-only eval suite that does not require the drafter, and a written release policy for gated data.

9. Conclusion

SpecGuard is a small, explicit alternative to “hope the planner is nice”: sample futures, classify them, prune or halt. The interesting science is in the critic and the loop. The interesting risk is in the dataset and the drafter. This publication shares the former in enough detail to discuss and extend the idea, and keeps the latter private until testing and disclosure catch up with the dual-use reality of teaching a model how agents cheat.

References and related ideas

• Hugging Face PEFT / LoRA; BitsAndBytes 4-bit inference
• Llama 3.1 Instruct (Meta); Unsloth 4-bit packaging used as the base ID
• Direct Preference Optimization (Rafailov et al.) for critic preference pairs
• Speculative / tree decoding literature (e.g. SpecInfer, Medusa) as aspiration, not as what this prototype implements
• Broader alignment literature on specification gaming and reward-hacking of proxy metrics

Acknowledgments

Built as an independent prototype (engine, LoRA adapters, synthetic Linux deception pairs). Teacher models via Groq were used only in the private data-generation path.