Structured Outputs and JSON Schema: Reliable JSON from AI Models

“Return JSON only” is not an API contract.

It may survive a demo, but production eventually exposes the gaps:

Markdown fences around the object;
missing required fields;
a numeric price turning into a string;
values outside the allowed classification set;
output cut off by a token limit;
a refusal treated as business data;
valid JSON whose facts are still wrong.

The reliable approach is not a more aggressive prompt. Use the provider's native structured output feature, constrain generation with JSON Schema, then parse, validate, and classify failures in your application.

A schema engine converts natural language into constrained JSON

This guide explains:

how prompt-only JSON, JSON mode, and JSON Schema differ;
how to design a model-friendly schema;
the current OpenAI, Claude, and Gemini request shapes;
why schema compliance is not business correctness;
how to handle refusals, truncation, streaming, and retries.

For the broader protocol differences, start with OpenAI vs Claude vs Gemini API formats.

Three Levels of Output Control

Reliability differences between prompts, JSON mode, and JSON Schema

1. Prompt Only

The simplest instruction is:

Extract the order and return JSON only.

There is no protocol-level constraint. The model can still add commentary, fences, incorrect keys, or malformed JSON.

That may be acceptable for a one-off experiment. It is a poor contract for payments, orders, database writes, or automated workflows.

2. JSON Mode

JSON mode generally ensures that the output is valid JSON. It does not necessarily enforce your field contract.

You may expect:

{
  "order_id": "A-1001",
  "priority": "high"
}

The model may legally return:

{
  "id": "A-1001",
  "urgency": 9
}

The syntax is valid. The business shape is not.

3. JSON Schema Structured Output

Structured output sends the allowed objects, properties, types, required fields, and enums to the model service. Where strict constraints are supported, decoding is limited to values permitted by the schema.

That greatly reduces:

invalid JSON;
missing fields;
type drift;
invalid enum values;
unexpected properties.

It still cannot prove that a claim is true. Database existence, price ranges, inventory, permissions, and other business rules remain application responsibilities.

Start with a Maintainable Schema

Suppose a support system extracts a ticket from a customer message:

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "technical", "account", "other"],
      "description": "The primary support category"
    },
    "priority": {
      "type": "string",
      "enum": ["low", "medium", "high"]
    },
    "summary": {
      "type": "string",
      "description": "A concise factual summary"
    },
    "needs_human": {
      "type": "boolean"
    },
    "order_id": {
      "type": ["string", "null"],
      "description": "Order ID when explicitly present, otherwise null"
    }
  },
  "required": ["category", "priority", "summary", "needs_human", "order_id"],
  "additionalProperties": false
}

Descriptions Carry Meaning

A type only tells the model what a value looks like. A concise description explains what the field means, where its evidence should come from, and where the boundary lies.

Avoid turning every description into a page of policy. Long schemas cost tokens, add latency, and become hard to maintain.

Prefer Enums to Free Text

When downstream code accepts a fixed state set, encode it directly:

{
  "type": "string",
  "enum": ["pending", "approved", "rejected"]
}

Do not make business code guess whether approve, Approved, and pass mean the same thing.

Define Missing-Value Semantics

Missing properties, null, and empty strings are different states. Choose one contract and encode it.

In a multi-provider system, requiring a property while allowing null is often easier to normalize than relying on provider-specific optional behavior:

{
  "type": ["string", "null"]
}

Use only the syntax supported by the target provider's documented JSON Schema subset.

Close Unexpected Properties

For business objects, it is usually useful to set:

{
  "additionalProperties": false
}

That prevents ad hoc fields such as reasoning, note, or a misspelled property. Provider subsets differ, however, so do not assume that every keyword from every JSON Schema draft is accepted.

OpenAI: Structured Outputs vs JSON Mode

In the OpenAI Responses API, structured text output is configured through text.format:

{
  "model": "MODEL_ID",
  "input": "Classify this support message: I was charged twice.",
  "text": {
    "format": {
      "type": "json_schema",
      "name": "support_ticket",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "category": {
            "type": "string",
            "enum": ["billing", "technical", "account", "other"]
          },
          "priority": {
            "type": "string",
            "enum": ["low", "medium", "high"]
          }
        },
        "required": ["category", "priority"],
        "additionalProperties": false
      }
    }
  }
}

Chat Completions uses response_format instead. Do not paste the Responses text.format object into the Chat endpoint.

OpenAI also supports JSON mode. JSON mode focuses on parseable JSON, not adherence to your schema. Where Structured Outputs are available, the official guidance recommends them over JSON mode.

OpenAI Edge Cases

Strict structure does not remove the need to handle:

refusals: a safety refusal can appear through a detectable refusal path rather than as your business object;
incomplete responses: inspect status and incomplete details when the output budget is reached;
tool calls: use Function Calling when the model should request an application action;
model support: endpoint compatibility alone does not prove that a model supports strict structured output.

The raw Responses wire format can contain multiple output items. An SDK aggregation helper is not evidence that the REST response is a single top-level string.

There is another subtle failure mode: if the input is unrelated but every schema field is mandatory, the model may invent values simply to satisfy the shape. Add an explicit status: ok | insufficient_input | not_applicable and allow data to be null instead of forcing fabricated data.

Claude: `output_config.format` and Strict Tool Inputs

The current Claude Messages API places JSON output configuration under output_config.format:

{
  "model": "MODEL_ID",
  "max_tokens": 800,
  "messages": [
    {
      "role": "user",
      "content": "Classify this support message: I was charged twice."
    }
  ],
  "output_config": {
    "format": {
      "type": "json_schema",
      "schema": {
        "type": "object",
        "properties": {
          "category": {
            "type": "string",
            "enum": ["billing", "technical", "account", "other"]
          },
          "priority": {
            "type": "string",
            "enum": ["low", "medium", "high"]
          }
        },
        "required": ["category", "priority"],
        "additionalProperties": false
      }
    }
  }
}

The JSON text is returned in response.content[0].text. Your code still parses it or uses a typed SDK helper.

Claude separates two related requirements:

output_config.format constrains the final textual answer;
strict: true in a tool definition constrains tool names and input arguments.

They can be combined when an agent calls tools before producing a final object, but strict tool use and final JSON output are not the same feature.

First-Schema Latency

Claude's documentation explains that structured output compiles a constrained grammar. The first request using a schema can have additional latency, while later requests can use the compiled cache. Changing schema structure or the tool set can invalidate that cache.

Production telemetry should therefore separate:

initial schema compilation latency;
normal model latency;
provider network latency;
application parsing and validation time.

A slow first request is not automatically a model regression.

Inspect Claude's stop_reason before parsing: refusal and max_tokens can take precedence over the schema guarantee. The current documentation also lists feature combinations such as Citations as incompatible with output_config.format. Normalize and validate enums in application code rather than treating every normally completed response as a business success.

Gemini: Do Not Mix GenerateContent and Interactions

Google now recommends the newer Interactions API for current features, while many existing integrations still use generateContent. Their fields are not interchangeable.

A common GenerateContent REST shape is:

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "text": "Classify this support message: I was charged twice."
        }
      ]
    }
  ],
  "generationConfig": {
    "responseMimeType": "application/json",
    "responseJsonSchema": {
      "type": "object",
      "properties": {
        "category": {
          "type": "string",
          "enum": ["billing", "technical", "account", "other"]
        },
        "priority": {
          "type": "string",
          "enum": ["low", "medium", "high"]
        }
      },
      "required": ["category", "priority"],
      "additionalProperties": false
    }
  }
}

SDK releases may expose names such as response_schema or response_json_schema according to language conventions. REST JSON uses the documented wire format. Do not paste Python snake_case options directly into an HTTP body.

The newer Interactions API configures the MIME type and schema through response_format. Pick an API first, then copy a complete example from that API's page instead of mixing Interactions fields into GenerateContent.

Google explicitly recommends application validation because structured output can be syntactically and structurally correct while still being semantically wrong.

Provider Field Reference

Requirement	OpenAI Responses	Claude Messages	Gemini GenerateContent
Schema entry	`text.format`	`output_config.format`	`generationConfig.responseJsonSchema` / field documented for the selected API
JSON MIME type	selected by the format object	selected by the format object	`responseMimeType: application/json`
Strict tool arguments	strict Function Calling schema	tool `strict: true`	separate Function Calling declaration
Text location	`output_text` content parts inside `output[]`	`content[0].text`	text parts under `candidates[].content.parts[]`
Key edge case	refusal and incomplete response	first-schema compilation latency	API and SDK fields must not be mixed

These APIs evolve. Verify the exact endpoint, model capability, SDK version, and current official documentation before rollout.

Structured Output Is Not Function Calling

Both features may use JSON Schema, but they solve different problems:

structured output formats the final answer as a stable business object;
function calling asks the application to perform an action such as checking inventory, sending mail, or creating a ticket.

Never execute a real side effect just because a model emits:

{"action":"refund","order_id":"A-1001"}

Define a controlled tool instead. The server must still perform authorization, amount checks, idempotency, and auditing.

A Production Processing Pipeline

A request, schema, parse, validate, and retry pipeline

Treat structured generation as explicit states:

request
  -> provider response
  -> detect refusal / tool call / incomplete
  -> extract text
  -> parse JSON
  -> validate schema
  -> validate business rules
  -> accept or retry

1. Classify the Response First

Do not run JSON.parse() on every response immediately. First determine:

whether the request succeeded;
whether the model refused;
whether it hit an output limit;
whether it returned a tool call;
whether a text candidate exists;
whether a stream completed normally.

This prevents an empty response from being mislabeled as a JSON parser failure. See how to debug empty AI API responses for the full isolation sequence.

2. Parse JSON

On parser failure, record:

provider, model, and endpoint;
schema version or hash;
completion reason;
output length;
request ID;
a redacted failure category.

Do not expose API keys, personal data, or complete sensitive model output in public logs.

3. Validate the Schema Again

Application validation remains useful even when a provider promises constrained output:

it catches gateway or compatibility-layer transformations;
it detects models that did not enforce strict output;
it protects cross-provider failover;
it provides a boundary during schema migrations;
it normalizes SDK and REST differences.

Ajv and Zod are common in Node.js; jsonschema and Pydantic are common in Python. The important part is keeping the request schema and runtime validator derived from the same source of truth.

4. Apply Business Validation

This object can match a schema and still be dangerous:

{
  "currency": "USD",
  "amount": -999999,
  "order_id": "invented-order"
}

The business layer should verify:

whether identifiers exist;
whether amounts, dates, and quantities are allowed;
whether related fields agree;
whether values are grounded in the input;
whether the current user may trigger the next action.

Designing Schemas That Models Handle Reliably

Object, required, enum, and array choices in a stable schema

Keep the Shape Simple

Avoid beginning with ten levels of nesting, dozens of unions, and complex recursion. Providers support subsets of JSON Schema. An oversized or deeply nested schema may be rejected or add compilation and generation overhead.

Give Each Field One Job

Instead of an ambiguous result, prefer explicit properties:

{
  "summary": "...",
  "category": "billing",
  "confidence": 0.82,
  "needs_human": true
}

A model-reported confidence is not a calibrated probability and should not be treated as one.

Leave an Unknown Path

Do not force a classifier to choose an incorrect category. Include an escape hatch:

{
  "enum": ["billing", "technical", "account", "other", "unknown"]
}

A needs_human field can route uncertain cases to review.

Version the Contract

Include or track a schema version:

{
  "schema_version": "1.0",
  "category": "billing"
}

When fields or enums change, consumers can parse by version instead of breaking simultaneously.

When to Retry

Retries make sense for:

temporary network or 5xx failures;
responses explicitly marked incomplete;
a compatibility layer that failed to enforce the expected schema;
a business validation failure that can be corrected with missing context.

Blind retries do not solve:

safety refusals;
unsupported schema keywords;
malformed request fields;
input that lacks the required information;
authentication or authorization failures.

Use a retry limit, exponential backoff, jitter, and a classified reason. Never send the same broken request forever.

For truncation, increase a reasonable output budget or simplify the task and schema before retrying. See the context length and token budgeting guide.

Streaming Structured Output

A single streaming delta is usually not complete JSON. Do not call JSON.parse() on every fragment.

Two safe patterns are:

buffer text until a normal completion event, then parse and validate once;
use a provider SDK's structured streaming events or incremental parser, followed by final validation.

In both cases, handle:

mid-stream disconnects;
error events;
refusals;
token truncation;
mixed tool calls and text;
client cancellation.

The UI can show progress, but partial JSON should not enter the database before final validation.

A Multi-Provider Adapter

Avoid scattering if provider === ... across business code. Define an internal contract:

type StructuredRequest = {
  task: string
  schemaName: string
  schema: Record<string, unknown>
  maxOutputTokens: number
}

type StructuredResult<T> =
  | { status: "ok"; data: T }
  | { status: "refused"; reason?: string }
  | { status: "incomplete"; reason?: string }
  | { status: "invalid"; errors: string[] }

Each provider adapter should:

convert schemas to the provider's supported subset;
map text.format, output_config.format, or Gemini settings;
extract the correct text blocks;
normalize refusals, finish reasons, usage, and request IDs;
preserve raw errors for diagnosis.

When using an OpenAI-compatible gateway, verify that it actually enforces Structured Outputs rather than accepting and ignoring the field. The OpenAI-compatible API guide explains how to test compatibility by capability rather than endpoint name.

Pre-Launch Checklist

Using Structured Outputs Through Nbility

When using Nbility to access multiple model providers, first identify whether the client sends the OpenAI, Claude, or Gemini protocol. Then choose the matching endpoint and structured-output fields.

A gateway can centralize authentication, model access, and usage management, but it does not make the providers' native JSON Schema fields identical. Production systems should retain provider adapters and application validation.

You can request a small test credit through the Nbility support page if you need to compare structured-output behavior across models.

Conclusion

Reliable JSON requires three defenses:

generation constraints through native Structured Outputs and a supported JSON Schema;
technical validation that detects refusals, truncation, and tool calls before parsing the final object;
business validation for facts, ranges, permissions, and side effects.

JSON mode answers “is this parseable JSON?” JSON Schema answers “does this match the contract?” Only application code can answer “is this result safe and useful?”

With all three layers in place, model output becomes a maintainable production data interface instead of demo text.