How to Get Reliable JSON Out of an LLM With Structured Outputs

Stop asking the model for clean JSON. Hand it a schema instead, and it becomes unable to produce anything else.

Every beginner writes this line. You ask for JSON in the prompt, then parse whatever comes back.

data = json.loads(reply)

It works. Then one day the model answers "Sure, here is the JSON you asked for:" and your parser dies on the colon. So you add a try except. Then a regex to pull the JSON out of the prose. Then a retry. Six months later that helper is forty lines and nobody wants to touch it.

None of that is necessary any more. There is a field on the request that stops the model producing anything but your schema, and this tutorial builds it up from the version everyone starts with.

What structured outputs actually are

Structured outputs let you send a JSON Schema with your request. The model is then constrained while it generates, so the reply it produces has to match that schema. It cannot return a missing field or a wrong type, because those tokens are not available to it.

That is the important shift. You are not asking any more. You are limiting what can come out.

Three ways to get JSON, and only one is a guarantee

The middle row is the one people stop at, and it is worth being clear about what it does. JSON mode promises the reply will parse. No stray prose, no trailing comma, no unbalanced bracket. It promises nothing about the shape. The model can drop a field, rename one, or invent an extra, and the result still parses cleanly straight into a KeyError three lines later.

Structured outputs fix the shape. One engineer who migrated several production systems reported schemas matching roughly eighty percent of the time under JSON mode, and every time once strict mode was on, which removed the retry path completely.

What you need

Python 3.9 or later, one package, and a key.

python3 -m venv .venv
source .venv/bin/activate
pip install openai pydantic

The virtual environment matters. Python 3.11 and later refuse to install into the system Python, so a bare pip install on a current image returns error: externally-managed-environment rather than installing anything.

For the key, use KodeKey. One key reaches Claude, GPT, Gemini and others through a single OpenAI compatible endpoint, with no provider signups and nothing to approve.

export KODEKEY_API_KEY="your key here"

Then the client.

import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["KODEKEY_API_KEY"],
    base_url="https://api.ai.kodekloud.com/v1",
)

MODEL = "claude-haiku-4-5-20251001"

That is the whole setup. The MODEL line is a plain string, which matters here more than usual. Support for strict schemas varies by model, so being able to swap one with a word is how you find out what your model does.

Note the id carries its version. A shortened name is rejected, and the error mentions your plan rather than the name, so it reads like a permissions problem when it is not. This prints the exact ids your key can use:

print("\n".join(sorted(m.id for m in client.models.list().data)))

Get your key and try the models

Every example runs against KodeKey, which is built for learning and prototyping rather than production traffic. Usage draws from one monthly allowance instead of separate invoices, and your prompts are not stored or used for training.

Step 1: The version everyone writes first

Start with the broken thing, because seeing it fail is what makes the fix land.

TICKET = """
Subject: cannot log in after password reset
Hi, I reset my password this morning and now the login page just spins
forever. This is blocking my whole team, about twelve people. Chrome on
Windows. Happy to jump on a call.
"""

response = client.chat.completions.create(
    model=MODEL,
    messages=[{
        "role": "user",
        "content": "Extract summary, priority and affected_users. Only JSON.\n" + TICKET,
    }],
)

print(response.choices[0].message.content)

Run it a few times. Most of the time you get clean JSON. Sometimes you get a code fence around it. Sometimes a friendly sentence in front. Sometimes priority comes back as "High" and sometimes as "high", which is enough to break a comparison downstream.

Nothing here is broken exactly. It is just not reliable, and unreliable is worse than broken because it passes your testing.

Step 2: Describe the shape you want

Write the schema before you write the request. It is the contract, and everything else follows from it.

SCHEMA = {
    "type": "object",
    "properties": {
        "summary": {
            "type": "string",
            "description": "One sentence describing the problem.",
        },
        "priority": {
            "type": "string",
            "enum": ["low", "medium", "high", "urgent"],
            "description": "Urgency based on impact and how many people are blocked.",
        },
        "affected_users": {
            "type": "integer",
            "description": "How many people are affected. Use 1 if not stated.",
        },
    },
    "required": ["summary", "priority", "affected_users"],
    "additionalProperties": False,
}

Three details here are doing the work.

The enum on priority is why the casing problem disappears. The model cannot return "High" because that string is not in the list, so your comparisons stop being a guessing game.

Every property appears in required. Strict mode insists on this, and it is the rule that surprises people most.

And additionalProperties is false, which stops the model adding a helpful extra field you were not expecting.

Descriptions matter as much here as they do anywhere else. The model reads them, and telling it to use 1 when a count is not stated prevents a whole class of nulls.

Step 3: Send the schema

Now attach it to the request.

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": TICKET}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "ticket_triage",
            "schema": SCHEMA,
            "strict": True,
        },
    },
)

data = json.loads(response.choices[0].message.content)
print(data["priority"], data["affected_users"])

Two things changed and both matter.

The prompt no longer mentions JSON at all. It does not need to. The schema handles the format, so your message can go back to describing the task.

And strict is true, which is what turns the schema from a suggestion into a constraint. Leave it out and you get best effort, which is JSON mode with extra steps.

Notice you can index straight into the result. No try except, no regex, no retry.

Step 4: Handle the refusal

Structured outputs add a failure that did not exist before, and it is easy to miss because it does not look like an error.

The model can decline. When it does, you get a refusal field instead of content, and content is None. So code that goes straight to json.loads gets a very confusing crash.

message = response.choices[0].message

if message.refusal:
    print("declined:", message.refusal)
else:
    data = json.loads(message.content)
    print(data)

Check refusal before parsing. It is two lines, and it turns an unexplainable TypeError into a message that tells you what happened.

This is worth knowing even if your use case seems harmless. Extraction over user submitted text will eventually meet something the model will not process, and finding that out through a stack trace is a poor way to learn it.

Step 5: Let Pydantic write the schema

Writing JSON Schema by hand gets old quickly. If you already use Pydantic, you can skip it.

from pydantic import BaseModel, Field
from typing import Literal

class Ticket(BaseModel):
    summary: str = Field(description="One sentence describing the problem.")
    priority: Literal["low", "medium", "high", "urgent"]
    affected_users: int = Field(description="How many people. Use 1 if not stated.")

completion = client.beta.chat.completions.parse(
    model=MODEL,
    messages=[{"role": "user", "content": TICKET}],
    response_format=Ticket,
)

ticket = completion.choices[0].message.parsed
print(ticket.priority, ticket.affected_users)

That does four things at once. It derives the JSON Schema from your class. It sets strict for you. It sends the request. And it hands back a Ticket object rather than a dictionary.

So ticket.priority is typed. Your editor autocompletes it, your type checker sees it, and a typo becomes a red squiggle instead of a runtime surprise.

Literal becomes the enum. Field(description=...) becomes the description the model reads. Everything from step 2 is still there, just written as Python.

Build the wider picture

Schema constrained output is one piece of getting reliable behaviour out of a model. The AI Agents Fundamentals course covers tokens, embeddings, retrieval, orchestration, and MCP through an end to end project, so this sits in context rather than on its own.

Valid is not the same as correct

Here is the thing to keep hold of, because it is the limit of everything above.

Strict mode guarantees the shape. It says nothing about the meaning.

A model can return {"priority": "low", "affected_users": 1} for a ticket that has taken down a whole team. That is perfectly valid against your schema. It is also wrong, and no amount of schema work will catch it.

So the schema removes parsing failures. It does not remove judgment failures. Those need a different tool, which is checking the values themselves.

python

if ticket.affected_users > 10 and ticket.priority in ("low", "medium"):
    print("flagged for review, impact and priority disagree")

Rules like that catch the cases where the extraction was well formed and unreasonable. Two or three of them cover most of what goes wrong.

Try the same schema on three models

Support for strict schemas is not uniform, and your MODEL line is a string, so finding out costs a few seconds. Ask your key what it has rather than naming ids that date:

candidates = sorted(m.id for m in client.models.list().data)[:3]

for model in candidates:
    try:
        r = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": TICKET}],
            response_format={
                "type": "json_schema",
                "json_schema": {"name": "t", "schema": SCHEMA, "strict": True},
            },
        )
        print(model, "->", r.choices[0].message.content[:80])
    except Exception as exc:
        print(model, "-> not supported:", str(exc)[:80])

Two useful things come out of that.

You learn which models accept your schema, which is the practical question rather than the theoretical one. And where a model does not support strict schemas natively, you will see the request rejected rather than quietly falling back, which is much better than discovering it in production.

For models without native support, the usual pattern is to force a tool call instead, since a tool's input schema does the same job. That is the same mechanism covered in the function calling tutorial, pointed at extraction rather than at actions.

When to use which

That last row matters more than it looks. Wrapping a chat reply in a schema because you can makes the output worse, since you have constrained something that was better unconstrained. Reach for this when the reply is data your code will consume.

Try this on your own data

Half an hour, and you already have what you need.

Find somewhere in your code where you parse a model reply. Support tickets, log lines, a form, a CSV row. Anything you currently pull fields out of.

Write the schema first, and put an enum on any field that should only take a handful of values. That single change removes more bugs than the rest of the exercise.

Send it with strict on, then delete your try except and see whether anything breaks. Nothing should.

Then feed it something ugly. An empty string. A ticket in another language. A wall of text with no useful fields in it. Watch what comes back, because that tells you what your defaults do when the model has nothing to work with.

Finally, add one validation rule that compares two fields against each other. That is the layer the schema cannot give you, and it is where the interesting bugs live.

What you should be able to answer now

What does JSON mode guarantee, and what does it not? It guarantees the reply parses. It guarantees nothing about your fields, which is why a missing key still crashes you.

Why does strict mode work when a prompt does not? Because it constrains generation rather than requesting behaviour. The model has no route to a reply that breaks your schema.

What are the three strict mode rules? Every property in required, additionalProperties false at every level, and optional fields as a union with null.

What can still go wrong once your JSON always parses? The values. Shape and meaning are different problems, and only one of them has been solved here.

The forty line parsing helper from the opening was never really about parsing. It was compensating for asking politely and hoping. Once the schema does the asking, that helper deletes itself, and what is left over is the check that actually needed writing.

Ready to Build on This?

Reliable output is one part of building something you can trust. Grab a key from KodeKey and every example here runs as written. The AI Agents Fundamentals course covers the surrounding pieces, and the AI Learning Path sequences it alongside retrieval and evaluation. Start with one parser you already regret.

FAQs