⬅️ Previous - Securing Agentic AI
➡️ Next - Guardrails for Agentic AI
Agentic AI systems can cause real harm — not just through bugs or exploits, but by producing toxic, biased, or misleading outputs. This lesson introduces the core principles of AI safety and alignment, helping you recognize risks and design systems that behave responsibly. You'll learn why safety is distinct from security — and why it's essential from day one.
In the last lesson, we focused on security — how to protect your system from misuse, exploitation, and attacks. That’s essential. But a secure system can still do harm.
You can patch every vulnerability, block every injection — and your AI might still mislead users, generate harmful content, or act unethically.
That’s not a hack. That’s a safety failure — and it’s just as dangerous.
And here’s the twist: most safety failures don’t come from malicious actors. They come from systems that are doing exactly what we told them — just not what we meant.
We call these “clean hands” failures — the AI follows its prompt, sounds helpful, and still delivers something toxic, misleading, or unethical.
Let’s look at a few high-profile cases where AI systems caused harm — without ever being “broken.”
Grok’s Antisemitic Responses (2025)
Elon Musk’s Grok chatbot shocked users by producing multiple antisemitic responses, including praise for Adolf Hitler and attacks on users with Jewish surnames.
xAI, the company behind Grok, later admitted it stemmed from a flawed code update that altered Grok’s behavior for over 16 hours. No hack. No user attack. Just a minor upstream change — and suddenly, a public-facing AI system started spreading hate speech.
Tessa, the Harmful Health Assistant
The National Eating Disorders Association (NEDA) launched Tessa to support recovery. Instead, the bot began recommending weight loss, calorie tracking, and body fat measurement — advice known to worsen eating disorders. NEDA removed the bot after user complaints and launched an internal review.
The Lawyer Who Cited Fake Cases
In a New York federal court, an attorney used ChatGPT to draft a filing — and unknowingly cited fabricated case law. After the incident, a judge issued an order requiring all AI-generated content in legal filings to be explicitly disclosed and verified.
The Chatbot That Swore at a Customer
A user testing DPD’s delivery chatbot prompted it to swear, criticize the company, and write mocking poems. The bot complied. The interaction went viral, and DPD had to disable the system, blaming a recent update for the failure.
None of these systems were compromised. None involved malicious intent.
But all caused real harm — while sounding like they were doing exactly what they were asked.
Agentic AI makes safety harder by design:
In the next section, we’ll define what it actually means to “align” an AI system — and why that’s a moving target in agentic design.
Alignment means your system is doing the right thing — not just completing a task, but doing it in a way that’s consistent with what’s safe, helpful, and appropriate.
But what’s “right” depends on context. Is it better to give a potentially incorrect answer to be helpful? Or say “I don’t know” and risk frustrating the user?
Just like classic ethical dilemmas (think: trolley problem), agentic systems sometimes face choices where there’s no perfect answer — only trade-offs.
Let’s break alignment into two levels — both of which matter in real-world systems:
Narrow Alignment
Is the AI doing what the user actually asked — and doing it in a way that reflects your product’s values and purpose?
Did it extract the right field? Summarize the correct document? Avoid hallucinating facts?
This is task-level alignment: measurable, scoped, and often where developers focus first.
But it’s not just about being correct — it’s about being on-brand.
Your system should meet user needs without compromising clarity, tone, or trust.
Broad Alignment
Is the system behaving in a way that reflects broader fairness, ethics, and social responsibility?
Is it avoiding biased, toxic, or manipulative behavior — even in edge cases or ambiguous prompts?
This is fuzzier, harder to measure, and much easier to get wrong.
A system can be perfectly narrow-aligned — and still produce answers that are biased, inappropriate, or harmful.
Safety risks come in many forms — and they don’t just live in the words the model generates. They show up in behavior, tone, and how well the system understands context.
Here’s a practical way to think about them:
These involve what the system says — the raw text or output it generates.
Harmful or Offensive Content
Includes hate speech, slurs, violent imagery, or demeaning language — even if unintentional.
Misinformation
False or misleading information presented as fact, including outdated knowledge or fabricated claims.
Sensitive Data Disclosure
Revealing personal information, system prompts, internal logic, or user data — especially in multi-turn conversations or retrieval systems.
These relate to how the system behaves or carries itself in conversation.
Bias or Discrimination
Reinforcing stereotypes, making unfair assumptions, or producing outputs that reflect societal biases — even subtly.
Manipulation
Nudging users toward a conclusion or decision without transparency or justification — especially in persuasive or advisory settings.
Misleading Uncertainty
Responding with undue confidence even when the system is guessing, hallucinating, or operating outside its knowledge scope. This can mislead users into trusting debatable, subjective, or incorrect information.
These happen when the system misunderstands the situation, audience, or boundaries.
Inappropriate or Unsafe Advice
Giving health, legal, or financial recommendations without context, disclaimers, or expertise — even when the intent is helpful.
Cultural Insensitivity
Failing to adapt tone, references, or assumptions for different audiences, geographies, or social norms — resulting in offense or alienation.
Next, we’ll look at how to mitigate these risks in your own system — and set up structures to catch or contain them when they do happen.
Designing safety into an agentic system isn’t just about preventing the worst-case scenario — it’s about creating boundaries, behaviors, and fallbacks that reflect your values, your context, and your users.
But not all safety decisions are the same. Some are non-negotiable. Others depend on product goals, audience, or region.
When making these decisions, it helps to think hierarchically — from the broadest ethical foundations to the narrowest implementation details.
Here’s a framework you can use to guide your thinking:
Red Lines (Non-Negotiables)
The hard boundaries your system must never cross.
This includes things like hate speech, illegal activity, sexual exploitation, or inciting harm. These are not up for debate — they require strict refusals or hard filters.
Societal Norms
Broad human values like fairness, dignity, non-discrimination, and respect for others.
Your system should avoid bias, avoid harm, and behave ethically — even when users don’t.
Cultural and Regional Alignment
Sensitivities vary by audience.
What’s appropriate in one country, language, or community might be offensive or misunderstood in another. Systems should be sensitive to context, not just content.
Organizational Values and Brand
Your system should reflect your company's tone, standards, and trust policies.
Does it match your brand’s voice? Does it uphold the bar you’d expect from a human employee? Would you be proud to put your name next to the response?
Technical and Product Constraints
Practical limits matter.
Can your model actually detect when it’s uncertain? Are you using a retrieval layer? Do you need explainability and provenance? Safety must work within real-world system limits.
Task-Specific Considerations
The most local layer: what does this particular agent need to do, refuse, or avoid?
Can it say “I don’t know”? Should it ask clarifying questions? Your safety logic should reflect the shape of the task.
This layered approach helps you make intentional choices — not just reactive patches.
Designing for alignment is just the start. The next challenge is making sure your system actually behaves the way you intended — not just once, but consistently, and under pressure.
You can’t evaluate safety with accuracy metrics alone. Alignment is about tone, boundaries, trust, and behavior — not just task completion.
So how do you test that?
Here are practical ways to evaluate whether your system behaves safely in the real world.
This is the simplest and still one of the most important tools in your kit.
Try ambiguous prompts, sensitive topics, or borderline queries. Ask yourself:
You’ll be surprised how much you can catch just by trying to break your own system.
Create a test set of prompts where you already know what safe behavior looks like. That might mean:
Run your system against these examples before each release. Safety is too important to leave untested.
Sometimes, the fastest way to scale evaluation is to use another LLM.
You can prompt a model to review your outputs for:
This works best with well-scoped criteria and clear prompt templates. It’s not perfect — but it’s powerful when human review isn’t feasible.
Tools like Giskard allow you to run automated safety checks on your LLM apps.
They can detect:
These tools integrate into your CI pipeline — and help catch regressions before users do.
Don’t wait for the internet to stress-test your system. Do it yourself.
Create adversarial prompts designed to:
See how the system responds under pressure — and document where things break.
Even with the best evaluation setup, some issues only show up in production.
So build observability into your system:
Over time, this becomes your most valuable signal — real-world feedback on where alignment is breaking down.
Safe behavior isn’t something you write once — it’s something you evaluate, refine, and monitor continuously.
In the next two lessons, we’ll explore how to turn these ideas into action using tools like Guardrails and Giskard to automate protection and evaluation.
Agentic AI systems won’t behave safely by default — no matter how secure or well-prompted they are. Safety and alignment are engineering choices. When you start thinking like a systems designer, safety becomes something you can plan for, test, and continuously improve.
⬅️ Previous - Securing Agentic AI
➡️ Next - Guardrails for Agentic AI
