Structured Outputs: Making AI Return Useful Data
Learn how apps get reliable, machine-readable data out of a model, and why asking politely isn’t enough.
Chat answers are written for people. But much of what apps want from a model is meant for programs: an order to store in a database, fields to fill in a form, a list of tasks to create, a yes-or-no to act on.
Programs are fussy readers. They need exact field names, the right types (a number, not the word “three”), and nothing extra around the data. The usual format is JSON: labelled fields in curly brackets, the same format the API requests from earlier use.
There are three increasingly reliable ways to get it:
- Just ask. “Extract the order details.” You get a friendly paragraph. Lovely for people; useless for a program.
- Ask for JSON. Describe the fields and say “reply only in JSON”, ideally with an example. Most replies are fine, but at scale the cracks show: a cheerful “Sure! Here’s the JSON:” in front, a missing comma, a field called
qtyinstead ofquantity. A program reading a million replies will hit all of them. - Enforce a schema. Give the API a formal description of the exact shape you need, a schema. With structured output enabled, the provider only lets the model pick tokens that keep the output valid. Remember sampling in Track 2: at each step, every token that would break the schema has its probability set to zero. The result is valid JSON of the right shape every time.
One warning: valid isn’t the same as correct. A schema guarantees the shape, not the truth. The model can still put the wrong address in a perfectly formed field, so apps still check the values that matter, and ask a human when in doubt.
Key takeaways
- Apps often need data for programs, not prose for people, usually as JSON.
- Asking for JSON works most of the time, but at scale stray text and wrong fields will appear.
- Structured output enforces a schema while generating, by ruling out tokens that would break it.
- A valid shape doesn’t guarantee correct values: important fields still need checking.
Check your understanding
Go deeper (more technical)
Schemas are usually written in JSON Schema, a standard way to describe field names, types, required fields and allowed values. Several providers support schema-enforced output, and open-source engines (Track 4) do the same with “grammar-constrained decoding”, compiling the schema into a filter over the vocabulary.
A related feature, tool calling (Track 7), uses the same idea: the model fills in the arguments of a function, as JSON matching a schema, and the app runs the function.
The simulation’s runs are scripted to show realistic failure types.