JSON Structured Output on EXAIV: Getting Responses You Can Parse Reliably

"Return the result as JSON" is an instruction, not a guarantee. A model asked for JSON will usually produce JSON, and the ten percent of the time it does not — a trailing comma, a fenced code block, an unescaped quote inside a string — lands in your parser as an exception at exactly the wrong moment.

Structured output replaces the instruction with a constraint. You declare the schema, the request is only permitted to produce output that fits it, and the response becomes something your code can consume without hoping.

The three response formats

json_object is a real improvement over asking in prose, and it is still the right choice when the shape of the object can vary. For a stable contract, json_schema is better because the request documents the contract instead of leaving it implicit.

Defining a schema

{
  "model": "gpt-6-sol",
  "messages": [
    {"role": "user", "content": "Review this product description and rate it.\n\n'Excellent build quality, ships fast, slightly heavy.'"}
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "product_review",
      "strict": true,
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "verdict": {
            "type": "string",
            "enum": ["excellent", "good", "acceptable", "poor"]
          },
          "summary": {
            "type": "string"
          },
          "pros": {
            "type": "array",
            "items": {"type": "string"}
          },
          "cons": {
            "type": "array",
            "items": {"type": "string"}
          },
          "score": {
            "type": "number"
          }
        },
        "required": ["verdict", "summary", "pros", "cons", "score"]
      }
    }
  }
}

Two requirements are easy to trip over in strict mode:

• additionalProperties: false on every object. Without it, strict mode is rejected.
• Every property listed in required. Strict mode expects a complete object, so optional fields are expressed as nullable types rather than as absent keys.

If the request is rejected, the error body names the offending part of the schema. That message is more precise than most schema validators and is worth reading carefully.

Reading the response

import json

response = client.chat.completions.create(
    model="gpt-6-sol",
    messages=messages,
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "product_review",
            "strict": True,
            "schema": REVIEW_SCHEMA,
        },
    },
)

content = response.choices[0].message.content
data = json.loads(content)

The response body is a JSON string containing your JSON, so it is parsed once at the top level. The schema governs the inside; the outer envelope is still the normal chat completion.

What a schema cannot express

A schema constrains shape. It cannot express meaning, and it cannot express most of the rules that make a response correct.

Truthfulness. A schema cannot say the summary must be faithful to the input. A model can produce a perfectly valid object that misrepresents the document.

Coherence across fields. A schema can require that summary and score both exist. It cannot require that the score agrees with the summary.

Ranges inside a string. A numeric field can be bounded by a schema, but a string field that encodes a number cannot.

Completeness. Required fields can be present and empty. An empty string satisfies a string schema.

This is why schema validation is a floor rather than a ceiling. It eliminates a whole class of malformed output — the class that breaks parsers and downstream code — and it says nothing about the class of output that is well-formed and wrong.

Where the distinction matters: for anything that triggers an action, a change, or a decision, add explicit business validation on top of the schema. For anything that only feeds a display, the schema is usually enough.

Validate before you trust it

A schema that fits guarantees the shape. It does not guarantee the content is sensible, and it does not guarantee a response exists at all.

Check these before acting on the result:

That last row is the one people skip. If the verdict must be one of four labels, use an enum — the schema then enforces it. If a numeric score must be between 0 and 10, the schema alone will not enforce it unless the format supports it; add the range check in code.

Failure modes and what to do about them

Truncation is the most common of these and the easiest to prevent: a JSON object cut in half fails to parse, and the fix is a larger max_tokens or a smaller requested payload.

A defensive parse helper

import json

def parse_structured(content, validator):
    if not content:
        raise ValueError("empty response")
    try:
        data = json.loads(content)
    except json.JSONDecodeError as e:
        raise ValueError(f"invalid JSON: {e}") from e
    return validator(data)

Wrap it so that every call site handles the three failure modes — empty, invalid, semantically wrong — in the same way. Then a malformed response becomes an ordinary error your application already knows how to log and report.

Structured output versus tool calls

Both produce machine-readable results, and they solve different problems:

If you only need data back, structured output is simpler and cheaper. If you need something done, you need tool calling. Using tool calling for a pure extraction task adds a round trip and an execution loop for no benefit.

Choosing the right schema type

Not every extraction needs a schema. Picking the simplest option that meets the requirement keeps requests cheap and avoids brittle validation.

Two questions settle it. Does a downstream program parse the result? If yes, you want a schema. Is the shape of the answer fixed across requests? If yes, you want a schema; if not, json_object is enough.

The cost of a schema is that the model must fit its answer into a fixed shape. For a summary with five known fields, that constraint is a benefit. For an open-ended analysis, forcing it into a schema usually produces fields that are technically present and semantically empty.

Handling large structured outputs

Structured output has a size limit that is easy to forget: the complete object has to fit inside max_tokens alongside everything else. A schema with many fields and a request for long text in each field can exceed the ceiling, and the failure is a truncated object that will not parse.

Three ways to avoid it:

• Estimate the size before you request it. Count the fields and reason about the longest plausible value in each. A schema with eight free-text fields can produce thousands of tokens.
• Raise the ceiling deliberately. A larger max_tokens is the direct fix, and it is bounded by the output rate.
• Split the work. Ask for the summary in one call and the supporting detail in another, then combine them in your code. This keeps each request inside its budget and lets each part be retried independently.

The split approach has a second benefit: the two calls can use different models. A short structured extraction is a cheap task, and the long free-text part is the one that needs the stronger tier.

A fallback that stays honest

Sometimes a schema-constrained call fails and you still need an answer. The fallback should degrade in a controlled way:

1. Retry once with the same request — a transient failure may resolve.
2. Retry with json_object instead of the schema, then validate locally against your own schema.
3. If validation fails, report the failure. Do not return a half-parsed object.

Returning an invalid object because the caller needs an answer moves the failure downstream, where it is more expensive to detect and harder to trace. Failing visibly at the boundary is the better behavior.

Structured output checklist

• response_format is set instead of asking for JSON in the prompt.
• Strict mode schemas use additionalProperties: false and a complete required list.
• Optional fields are expressed as nullable types.
• Closed sets use enum; numbers have explicit ranges.
• max_tokens is large enough for the complete object.
• content emptiness and finish_reason are checked before parsing.
• The parsed result is validated against business rules beyond the schema.
• A fallback path exists and degrades visibly rather than returning invalid data.

Further reading

• Function Calling Guide — when the model needs to act rather than just answer.
• Chat Completions Parameters Reference — where response_format fits among the other request fields.
• Error Handling and Retry Guide — handling a structured call that fails.

FAQ

Related reading

• Function Calling Guide
• Chat Completions Parameters Reference
• Streaming Responses Guide

Terminology and pre-flight checklist

Total parameters, active parameters and the KV cache are three different quantities. Total parameters are all the weights that must be loaded into memory; active parameters are the subset that computes a single token. KV cache is temporary memory that grows linearly with context length, so a 1M window is a ceiling rather than a free default. Quantization lowers the precision of stored weights to shrink files; it does not reduce the parameter count. A vision_config in a config.json describes architecture, not a wire format you may safely invent.

Before shipping anything built on these notes, run six checks. Re-read the numbers from the live source on the day you publish, because download counters move weekly. Open the live /models catalog and confirm the model ID you plan to call is still there. Open /pricing rather than copying a price into your application. Never hard-code a request body you have not validated against the official reference. Record the model ID, input and output token counts, and latency for a representative set of prompts, then keep that set unchanged so a later model swap produces a comparable reading. Read the licence on the model card before commercial use, since vendor-specific terms are not a permissive default. Add exponential backoff for retryable requests, set a timeout, and log the error class so that a 401, a 404 and a 429 never get the same treatment.