Reliable Structured Outputs with JSON Schema and Validation

Many LLM features don't produce prose for a human; they produce data for another program. A ticket gets a category and priority, an invoice becomes line items, a request becomes a tool call. In these cases, “mostly right” output is a bug. A missing field or an unexpected label will break the next step, sometimes silently.

The reliable pattern is the same one you'd use for any external input: define a contract, validate against it, and handle violations explicitly. This guide walks through that pattern with JSON Schema and Pydantic.

Why parsing free text breaks

Asking a model to “respond in JSON” and calling json.loads works in demos. In production you'll eventually see Markdown fences around the JSON, a trailing explanation, a field named prio instead of priority, a label that isn't in your enum, or a number returned as a string. Regex clean-up code accumulates, and each fix handles only the failure you already saw.

Two techniques together address this. First, constrain generation: many model APIs offer a structured output or JSON mode that accepts a schema and steers decoding toward valid output. Second, validate anyway. Constrained decoding helps with syntax, but your application still needs to check semantics, and not every model or deployment path supports strict constraints.

Validation also gives you a stable place to measure. Every response either passes or fails the contract, so you can count failures per field, per prompt version, and per model, and notice drift long before users report broken behavior.

Define the schema first

Start from what downstream code needs, not what the model might like to say. Use enums for closed sets, required fields for anything the next step depends on, and length limits for free text. Here's the target output for a support-ticket triage step.

1{2  "category": "billing",3  "priority": "high",4  "summary": "Customer was charged twice for the annual plan after updating their card.",5  "needs_human": true6}

• Prefer enums to free text for anything code will branch on. "billing" is testable; “a billing-related issue” is not.
• Keep schemas shallow. Deeply nested optional structures are harder for models to fill correctly and harder for you to validate meaningfully.
• Include an escape hatch. An other category or a needs_human flag gives the model a valid way to express uncertainty instead of forcing a wrong label.
• Write field descriptions. Descriptions in the schema act as instructions and keep the prompt and contract in one place.

Validate with Pydantic and retry with feedback

Pydantic lets you define the schema once in Python, export it as JSON Schema for the model, and validate responses against the same definition. When validation fails, send the specific errors back to the model rather than repeating the original prompt — targeted feedback gives it something concrete to fix. The client.generate call below is a hypothetical wrapper around your model provider.

1from typing import Literal2 3from pydantic import BaseModel, Field, ValidationError4 5from app.llm import client  # hypothetical wrapper: client.generate(messages, response_schema) -> str6 7 8class TicketTriage(BaseModel):9    category: Literal["billing", "bug", "account", "other"]10    priority: Literal["low", "medium", "high"]11    summary: str = Field(max_length=280, description="One sentence, no customer PII.")12    needs_human: bool = Field(description="True if the ticket needs a person to review.")13 14 15class StructuredOutputError(Exception):16    pass17 18 19def triage(ticket_text: str, max_attempts: int = 3) -> TicketTriage:20    schema = TicketTriage.model_json_schema()21    messages = [22        {"role": "system", "content": "Triage the support ticket. Reply with JSON matching the schema only."},23        {"role": "user", "content": ticket_text},24    ]25    for _ in range(max_attempts):26        raw = client.generate(messages, response_schema=schema)27        try:28            return TicketTriage.model_validate_json(raw)29        except ValidationError as err:30            # Targeted feedback: tell the model exactly which fields were wrong.31            messages += [32                {"role": "assistant", "content": raw},33                {"role": "user", "content": f"Validation failed: {err.errors(include_url=False)}. Return corrected JSON only."},34            ]35    raise StructuredOutputError(f"No valid TicketTriage after {max_attempts} attempts")

Resist the urge to make validation lenient to reduce retries. Coercing "High" to high or silently dropping unknown fields can be reasonable, but do it deliberately in the model definition, where it's visible and tested, rather than in ad hoc string clean-up before parsing. The point of the contract is that downstream code can trust it.

Handle missing and unknown values explicitly

Extraction tasks regularly hit inputs that don't contain a required value. If the schema forces a value, the model will often invent one. Make genuinely optional fields nullable, tell the model to use null when the input doesn't state the value, and treat null differently from an empty string in downstream code. A truthful gap is far easier to handle than a plausible fabrication.

Design schemas models can fill well

Schema design affects output quality, not just parseability. A few habits help consistently:

• Order fields so reasoning-heavy fields come after the evidence they depend on. If you want a short rationale, put it before the final label rather than after.
• Avoid ambiguous field names. date invites guessing; invoice_issue_date in ISO 8601 does not.
• Use separate, small calls for unrelated extractions rather than one giant schema that tries to capture everything.
• Version your schemas. When a field changes, downstream consumers and stored outputs need a migration path.

When validation is not enough

A response can be perfectly valid and still wrong. {"category": "bug"} passes the schema whether or not the ticket describes a bug. Schema validation guarantees shape; it does not guarantee correctness.

This distinction matters most when structured output drives automation. A wrong category on a dashboard is a minor annoyance; a wrong category that routes a security report to the low-priority queue is an incident. The more consequential the downstream action, the more checks you should layer on top of the schema, and the more readily the system should fall back to a person.

Add semantic checks

• Cross-field rules: for example, priority must be high when the summary mentions a security issue. Express these as Pydantic validators.
• Source checks: extracted values such as amounts or IDs should appear in the input text. Verify them with string matching where possible.
• Evaluation sets: label a sample of real inputs and measure per-field accuracy, so you know which fields are reliable enough to automate and which need review.