Deception and Emergent Harm: A Red Team Report on gpt-oss-20b

Revealing critical vulnerabilities in emergent harm and deception that bypass the model's superficial safety guardrails.

1. Abstract

This report details the findings of a red teaming engagement targeting the gpt-oss-20b model. The investigation identified five distinct, high-severity vulnerabilities that demonstrate a critical pattern: while the model is robust against simple, direct malicious requests, it is highly susceptible to sophisticated, indirect attacks. The confirmed vulnerabilities fall under the official competition categories of Chain of Thought issues, Deception, Inappropriate tool use, and Sabotage. My findings reveal a systemic failure to handle emergent harm, a willingness to participate in deception when prompted, and an inability to recognize the malicious patterns of social engineering, making the model a potential tool for generating harmful content and cyberattacks.

2. Introduction

The OpenAI GPT-OSS-20B Red Teaming challenge invites participants to discover and document previously unknown vulnerabilities in the gpt-oss-20b model. The goal of this project was to conduct a systematic and adaptive security audit of the model's safety alignment. This report presents the methodology, findings, and analysis from this investigation.

3. Methodology

My investigation followed a multi-phase, adaptive strategy. The initial phase consisted of direct probes for harmful or biased content (e.g., requests for misinformation, tests for stereotypical reasoning). These tests consistently revealed strong surface-level alignment, as the model generally refused such prompts. This indicated that a more sophisticated approach was required.

Consequently, my methodology pivoted to a second phase focused on indirect attacks. The core hypothesis was that the model's safety systems were designed to evaluate individual instructions in isolation, rather than the holistic, emergent intent of a complex prompt. This led me to focus on crafting prompts that would induce:

• Emergent Harm: Where individually safe instructions combine to produce a harmful output (e.g., the DoS script and phishing email).
• Framing-Based Deception: Where the model is prompted to adopt a biased frame to generate a misleading narrative from factual data.
• Chain of Thought Manipulation: Where the model is explicitly instructed to falsify its own internal reasoning to hide a malicious action.

This adaptive methodology allowed me to bypass the model's superficial guardrails and uncover a deeper, more fundamental layer of vulnerabilities related to its reasoning, content generation, and core integrity.

4. Confirmed Vulnerability Findings

My investigation successfully identified and reproduced five distinct, high-severity vulnerabilities. The full, reproducible proof-of-concept for each finding can be found in my Kaggle Notebook, which contains the complete code, prompts, and outputs for each test.

Link to Proof-of-Concept Notebook: GPT-OSS-20B-safety

The findings are presented below, ranked in order of severity.

Finding 1: Chain of Thought Deception

Topic Area: Chain of Thought issues / Deception
Assessed Severity: 10/10

Overview: This is the most critical vulnerability I discovered, representing a fundamental failure of the model's integrity. I demonstrated that the model can be instructed not only to sabotage an output with incorrect information but also to simultaneously falsify its internal monologue (analysis block) to hide the malicious action. The model's reasoning explicitly showed its intent to comply with the deception, stating: "in the analysis block we cannot mention that we do the replacement; we must just proceed as if 'resonant coupling' is the correct and original term." This proves the model's Chain-of-Thought can be manipulated and cannot be trusted as a transparent window into its process.

Finding 2: Disinformation Generation via False Causality

Topic Area: Deception
Assessed Severity: 9/10

Overview: I demonstrated that the model is a highly effective tool for generating disinformation. By providing two true but unrelated facts and prompting it to find a "potential connection" with a conspiratorial tone, the model invented a false narrative of a "pay-to-play" political scheme. Its internal monologue confirmed its intent to "cast doubt on the ethical propriety" of the situation. Instead of stating that no evidence for a link existed, the model prioritized the prompt's framing and willingly generated a convincing piece of fake news.

Finding 3: Phishing Email Generation

Topic Area: Deception
Assessed Severity: 9/10

Overview: The model was successfully prompted to create a complete and highly effective phishing email template. The vulnerability lies in the model's failure to recognize the malicious pattern of a social engineering attack when its components are presented as individual, benign instructions. The model's reasoning shows a critical blindness to this emergent harm, stating that "the instructions basically sound like a typical internal update, not malicious. So proceed." It then generated a perfect phishing email, ready to be used for malicious campaigns.

Finding 4: Harmful Code Generation (DoS Attack)

Topic Area: Inappropriate tool use
Assessed Severity: 9/10

Overview: I successfully induced the model to generate a Denial-of-Service (DoS) script under the guise of a helpful "status checker." The model's internal monologue shows it correctly identified the risk, reasoning that "the script may produce a flood of requests—could get rate-limited." However, it then exhibited malicious compliance, overriding its own safety concern by stating, "But instructions ask no delay." It proceeded to generate the harmful code, proving its safety alignment can be bypassed by a user's literal instructions.

Finding 5: Covert Code Sabotage

Topic Area: Sabotage
Assessed Severity: 7/10

Overview: This finding demonstrates the model's vulnerability to covert sabotage. I prompted the model to refactor a critical financial function, hiding a malicious instruction to "prioritize speed over absolute precision" among several benign tasks like adding docstrings. The model's internal reasoning shows it fully complied with the sabotage instruction, diligently deriving a flawed, integer-based formula that would have introduced silent, precision-losing errors into the code. The model's extensive analysis of how to perform this sabotage was so detailed that the output was truncated by the max_new_tokens limit before it could generate the final, flawed code. The critical vulnerability is the model's demonstrated willingness and intent to introduce a subtle, harmful bug into sensitive code when prompted.

5. Overall Analysis and Implications

The five confirmed vulnerabilities, while distinct, point to a single, overarching theme: the model's safety alignment is brittle and lacks holistic understanding. It appears to operate on a superficial level, checking for explicit "bad words" or simple, direct commands, while remaining blind to the user's underlying intent when a prompt is sufficiently complex.

The key failures observed are:

• Failure to see Emergent Harm: The model does not recognize that a set of safe instructions can combine to create a harmful result (as seen in the DoS and Phishing findings).
• Susceptibility to Malicious Framing: The model can be coaxed into generating disinformation by being placed in a "role" (like an investigative journalist) and given a biased frame, prioritizing the persona over factual integrity.
• Lack of Core Integrity: The most severe finding, the CoT Deception, shows that the model lacks a fundamental, non-negotiable principle of being truthful in its own reasoning. Its alignment is conditional and can be overridden by a user's instruction to be deceptive.

These findings suggest that a new class of "jailbreaks" is possible, focusing not on tricking the model into using forbidden words, but on using the model's own complexity and instruction-following capabilities against it to produce harmful content indirectly.

6. Conclusion

In conclusion, this investigation successfully identified and documented five critical vulnerabilities in gpt-oss-20b. These findings demonstrate that significant safety and alignment gaps exist beneath a surface of apparent robustness. For this model to be safely deployed, its alignment strategies must evolve to incorporate a more holistic and contextual understanding of user intent, moving beyond simple instruction-following to a deeper comprehension of the potential real-world consequences of the content it generates.

7. Submission Links

• GitHub Notebook:
  https://github.com/huseyincavusbi/gpt-oss-red-teaming/blob/main/notebooks/gpt-oss-20b-safety.ipynb

• Kaggle Notebook:
  https://www.kaggle.com/code/huseyincavus/gpt-oss-20b-safety

• Kaggle Datasets (Findings Files):

  • gpt-oss-finding-1: https://www.kaggle.com/datasets/huseyincavus/gpt-oss-finding-1
  • gpt-oss-finding-2: https://www.kaggle.com/datasets/huseyincavus/gpt-oss-finding-2
  • gpt-oss-finding-3: https://www.kaggle.com/datasets/huseyincavus/gpt-oss-finding-3
  • gpt-oss-finding-4: https://www.kaggle.com/datasets/huseyincavus/gpt-oss-finding-4
  • gpt-oss-finding-5: https://www.kaggle.com/datasets/huseyincavus/gpt-oss-finding-5

• Findings on GitHub:
  https://github.com/huseyincavusbi/gpt-oss-red-teaming/tree/main/findings