---

🏠 Home - All Lessons

⬅️ Previous - Advanced Prompt Engineering
➡️ Next - Function Chaining

---

TL;DR

In this lesson, you’ll learn two powerful approaches to generating structured output from LLMs: prompt-based formatting, which works with any model using clear instructions and examples, and model-native structured output, which enforces structure during generation for guaranteed format and type safety. You'll see how tools like Pydantic and LangChain help validate outputs, and why model-native output should be your go-to whenever possible for building reliable, chainable, production-ready LLM applications.

Why Structured Outputs Matter?

Imagine you're building a powerful AI assistant that helps users shop online. They describe a product "a stylish, waterproof smartwatch with heart rate tracking" and your assistant is expected to understand that, generate a product listing, and hand it off to a database or another service. But here's the problem: if the assistant replies with a paragraph like “This is a sleek smartwatch perfect for fitness enthusiasts...”—how do you reliably extract the product name, price, features, and category from that?

This is where things fall apart.

Large language models generate free-form text beautifully. They craft eloquent sentences, mimic natural conversation, and adapt to countless contexts. But when it comes to integrating their responses into applications that require structured, machine-readable data, things get tricky. Parsing that free-flowing language to extract the exact information you need - names, categories, prices, or feature lists - is a very hard task to do.

This is why structured output from LLMs is so important. By guiding the model to generate responses in a predefined format like JSON, dictionaries, or even Pydantic models, you transform its output from expressive text into dependable data. This makes it easier to chain outputs into functions, feed results into APIs, or store them cleanly in a database.

In this lesson, you’ll learn how to design prompts, use validation tools, and leverage model-native features to ensure your LLM doesn’t just talk - but talks in the language of structure.

Creating Structured Output

When it comes to generating structured data from an LLM, there are two primary approaches:

1. Regular Prompting with Format Hints – This involves guiding the model through carefully crafted instructions and examples. It's compatible with most LLMs and relies on strong prompting strategies and post-processing tools like parsers and validators.

2. Model-Native Structured Output – This newer approach is supported by certain advanced LLMs (like OpenAI’s GPT-4 or Anthropic’s Claude) and integrates structured output directly into the model’s generation process.

In this lesson, we’ll cover both methods: starting with regular prompting, which is compatible with virtually all LLMs, and then exploring model-native structured output, a more advanced approach supported by specific models and APIs that guarantees format compliance during generation.

---

Option 1: Regular Prompting with Format Hints

LLMs are extremely responsive to the structure and clarity of your instructions. By explicitly defining the output format and showing a realistic example, you significantly improve the model’s chances of producing consistent, machine-readable output.

Key Techniques for Effective Structured Prompts

1. Explicit Format Instructions: Clearly define the expected structure, whether JSON, XML, or another format.
2. Demonstrate with an Example: Models learn patterns best when they can imitate a well-formatted sample.
3. Be Precise and Predictable: Avoid unnecessary prose or ambiguous language; stick to a strict template.
4. Standardize Field Names and Types: Keep formatting consistent to reduce variability in the output.

Example: Prompting for JSON Output

Let’s revisit the product listing example from earlier. Here’s how you can prompt an LLM to generate structured JSON output:

json_prompt = PromptTemplate(
    input_variables=["product_description"],
    template="""
    Create a product listing based on this description: {product_description}
    
    Return the information as a JSON object with the following keys:
    - name: product name
    - price: suggested price as a number
    - features: array of key product features
    - category: product category
    
    Example format:
    {{
        "name": "Wireless Earbuds",
        "price": 79.99,
        "features": ["Noise cancellation", "Waterproof", "20hr battery life"],
        "category": "Electronics"
    }}
    """
)

This approach already offers a strong foundation. By carefully crafting prompts with clear formatting examples, you can achieve reliable, structured outputs from most LLMs even without special tooling. However, this method still carries some risk: the model might occasionally drift from the expected format, requiring post-processing or error handling.

Now, let’s look at the second method - model-native structured output - which avoids these issues entirely by having the model generate data that conforms to a predefined schema during the generation process. This technique is only supported by certain LLMs but offers strong guarantees for structure, type safety, and downstream reliability.

---

Option 2: Model-Native Structured Output

Imagine never having to worry about parsing the model’s response, handling missing fields, or fixing broken JSON. You ask for structured data—and the model simply gives it to you. No guessing. No retries. No duct tape.

That’s the promise of model-native structured output.

This approach doesn’t rely on clever prompt engineering or post-hoc parsing. Instead, it integrates structure enforcement directly into the model’s generation process. You define the schema (say, with a Pydantic model), and the model generates its output to match that schema at the decoding level, ensuring the structure is baked into the response as it's produced.

But there’s a catch: this only works with LLMs that explicitly support structured outputs, such as OpenAI’s GPT-4 or Anthropic’s Claude. Fortunately, tools like LangChain make it easy to tap into this capability, particularly using .with_structured_output(...). Here is a list of models supporting this approach

Let’s see what this looks like in action.

---

Example: Native Structured Output with LangChain

Here’s how to use LangChain’s structured output feature with OpenAI's gpt-4o-mini to generate product data that conforms to a strict schema:

from typing import List
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from dotenv import load_dotenv

load_dotenv()

class Product(BaseModel):
    name: str = Field("The name of the product.")
    price: float = Field("The product's price.")
    features: List[str] = Field("Product's features.") 
    category: str = Field("Product category. One of [Beverages, Dairy, Grocery]")

# Initialize the model
model = ChatOpenAI(model_name='gpt-4o-mini', temperature=0)

# Define the prompt
prompt = ChatPromptTemplate.from_template(
    "Generate product information for: {description}"
)

# Chain prompt, model, and schema
chain = prompt | model.with_structured_output(Product)

# Run the chain
result = chain.invoke({"description": "Two kilos of tomato"})
print(result.model_dump())

And here’s the output:

{
    "name": "Fresh Tomatoes (2 kg)",
    "price": 4.99,
    "features": [
        "High in vitamins and antioxidants",
        "Locally sourced and organic",
        "Perfect for salads, sauces, and cooking",
        "Freshly picked for optimal flavor",
        "Versatile ingredient for various dishes"
    ],
    "category": "Grocery"
}

No parsing. No ambiguity. No prompt engineering. Just clean, structured, type-safe data.

---

Why This Is a Game-Changer

Model-native structured output fundamentally shifts how we think about using LLMs in production systems. It brings:

• Guaranteed Format Compliance: The model won’t hallucinate fields, it must follow the schema.
• Minimal Error Handling: Since the output structure is enforced at generation time, you rarely need to retry or validate after the fact.
• Better Developer Experience: You work directly with typed objects (like Pydantic models) rather than fragile strings.
• Fewer Surprises: No more unexpected formats or missing fields in mission-critical applications.

Coding Example

In the following three videos, we’ll bring everything we’ve discussed into action—step by step.

Free-Form Output (No Structure)

We'll start by showing what happens when you don’t enforce any structure at all. You’ll see the kinds of inconsistencies and ambiguities that make it hard to use LLM outputs in real-world pipelines.

Prompt-Based Structured Output (Approach 1)

Next, we’ll walk through how to design effective prompts that guide the model to produce well-formatted JSON. We’ll also validate the output using tools like Pydantic and LangChain parsers.

Model-Native Structured Output (Approach 2)

Finally, we'll show how to generate structured outputs using model-native features, such as LangChain’s .with_structured_output(...). You’ll see how structure is enforced during decoding, eliminating the need for prompt engineering or post-processing.

Whether you're building a data extraction pipeline, an AI assistant, or a multi-step function chain, these techniques will equip you to generate clean, structured, and type-safe data—ready for production.

Key Takeaways

• Free-form LLM outputs are unreliable for chaining, parsing, or integration into structured systems.

• There are two main approaches to structured output:

  • Prompt-Based: Works with any LLM, relies on clear instructions and examples.
  • Model-Native: Requires model support (e.g., OpenAI, Anthropic) and enforces structure during decoding.

• Prompt engineering can guide models to produce structured JSON, but it still requires validation and error handling.

• Model-native structured output eliminates parsing errors and avoids the need for custom prompts—offering clean, type-safe data by design.

By using these techniques, you unlock the full potential of LLMs in real-world, production-grade applications.

---

🏠 Home - All Lessons

⬅️ Previous - Advanced Prompt Engineering
➡️ Next - Function Chaining

---