BLUF — treat a structured LLM response as a versioned API contract, not as “JSON the model happened to emit”. Syntax validation is necessary, but it does not establish that an order, case classification or routing decision is safe to execute. A production boundary needs schema validation, semantic validation, explicit refusal/incomplete-output handling, a transformation layer and versioned telemetry.
This distinction matters whenever an LLM response enters ERP, CRM, ticketing or workflow automation. A syntactically valid object can still contain an unknown customer, a disallowed state transition, a negative quantity, a currency that does not match the source document, or a confident-looking answer that was cut off. The downstream system should receive only a canonical command that has passed its own business rules.
The contract boundary: five checks, not one
OpenAI describes Structured Outputs as a way to constrain a response to a developer-supplied JSON Schema. That is useful: JSON mode alone does not guarantee a particular schema. It is not a substitute for application validation. The same API response can contain a refusal, and a response can be interrupted before completion; both must be first-class outcomes in the contract.
A practical boundary has five stages: (1) transport and provider response checks; (2) parse and JSON Schema validation; (3) semantic business-rule validation against authoritative systems; (4) transformation from an accepted contract version to a stable internal command; and (5) policy-based execution or human review. Keep the raw provider response, validation result and correlation ID, but do not let raw model JSON directly select a database write.
Response-to-downstream architecture diagram
Source document / user request → model request (schema version pinned) → provider response → refusal / incomplete branch → JSON Schema validator → semantic validator + authoritative lookup → version transformer → idempotent command queue → ERP/CRM/ticketing action → audit and metrics.
The boundary should be fail-closed for actions with side effects. A retry is appropriate for a transient network failure or rate limit; it is not evidence that an invalid business value will become valid. Re-prompting can be useful for extraction, but bound the attempt count and retain the failed candidate for review.
Schema validation and semantic validation answer different questions
JSON Schema defines the shape and constraints of JSON instances. In particular, object properties can be declared and extra properties can be disallowed with additionalProperties: false. Use this to make accidental fields visible and to require fields needed by the next service. Avoid assuming a closed object is automatically easy to extend: JSON Schema’s own guidance notes that additionalProperties applies within its subschema, which can complicate extension through composition.
Semantic validation asks whether the shaped data makes sense in the current business context. It belongs outside the model: resolve customer_id in the system of record; verify that requested quantity is within the permitted range; check a purchase-order’s approval state; reject a requested action the caller is not allowed to perform. Schema-valid is not business-valid.
Decision table: choose an action by failure class
| Condition | Safe system response | Evidence to retain | Do not do |
|---|---|---|---|
| Provider refusal or safety block | Return a typed REFUSED result; route to a human or alternate permitted process | provider status, policy route, request correlation ID | Parse an invented empty object |
| finish reason indicates length/incomplete output | Return INCOMPLETE; reduce scope or continue only under an explicit continuation design | finish reason, token limit, contract version | Execute a partial action |
| JSON Schema fails | Return INVALID_SHAPE; bounded repair/retry only for non-side-effect extraction | validator errors, raw response hash, model and schema version | Strip unknown fields silently |
| Semantic rule or authoritative lookup fails | Return REJECTED_BUSINESS_RULE; human review or correction workflow | rule ID, lookup version, rejected fields | Retry until the model changes a fact |
| Valid v1 response for a v2 consumer | Transform v1 → canonical command; emit deprecation metric | source version, transformer version, canonical version | Make every consumer parse every version |
Evolve schemas without breaking consumers
Version the external response contract explicitly: contract_name plus contract_version in the envelope, and pin that version in the model request. Prefer additive changes that retain old required fields until consumers migrate. For a breaking change—renaming an enum, changing a unit, splitting a field—introduce a new version and transform both versions into one internal canonical command. Run producers and consumers in parallel, then retire the old version only after measured traffic is zero or the agreed migration window closes.
Do not make the model infer which version a downstream consumer expects. The request, response envelope, validator and transformer should agree on a declared version. Contract tests need representative accepted, refused, incomplete and invalid fixtures for every supported version.
Failure modes that reach production
1. “Valid JSON” with a missing required business fact. Mitigation: distinguish UNKNOWN from a guessed value; make the semantic validator reject unknowns for executable actions.
2. A schema change that is technically additive but changes business meaning. Mitigation: version units, enum semantics and defaults; test the canonical command, not only parser success.
3. A retry duplicates a downstream update. Mitigation: place an idempotency key and source-document hash on the command queue; the executor, not the LLM, owns deduplication.
4. The provider returns a refusal or truncated response and the parser treats it as an exception. Mitigation: model these as typed outcomes with a safe UI and operational route.
5. The application accepts fields it does not understand. Mitigation: use closed schemas where suitable and alert on validator failures; never silently drop material fields.
Operate the contract as a risk control
Track validation/refusal rates by model, model release, schema version, tenant or document class, and workflow. A rising INVALID_SHAPE rate after a provider or prompt change is a release signal, not merely an application error. Track semantic rejection separately: it may indicate source-data quality, an incorrect prompt, a rule change or attempted misuse. NIST’s Generative AI Profile frames governance, pre-deployment testing, content provenance and incident disclosure as primary considerations; this boundary creates the evidence needed to investigate those events.
Apply the same discipline used in the RAG security operating model for proving retrieval decisions: record the inputs, policy decision, versions and outcome. If the workflow also retrieves enterprise content, treat that content as untrusted input and apply the controls described in the prompt-injection guidance for retrieved documents. Before an LLM is allowed to propose an action against protected data, enforce entitlements before retrieval as described in the RAG access-control pattern.
What this does not solve
Structured output does not make extracted facts true, eliminate prompt injection, prove an entitlement, or decide whether an automated decision is appropriate. It also does not remove the need for domain-specific limits, human escalation and end-to-end evaluation. A strict schema can reduce flexibility and increase migration work; for exploratory or advisory use cases, a lighter contract may be proportionate. The engineering question is where a model output becomes an operational command.
Implementation checklist
□ Define a small versioned response envelope. □ Pin schema and model versions per release. □ Validate JSON Schema before any transformation. □ Resolve material facts and permissions against authoritative services. □ Transform accepted versions into one canonical, idempotent command. □ Type refusal, incomplete and invalid outcomes. □ Log correlation IDs, versions and validator/rule results. □ Gate promotion on validation and semantic-rejection trends.
Sources
OpenAI — Introducing Structured Outputs in the API
JSON Schema — object reference
NIST AI 600-1 — Generative AI Profile
Next step
If an LLM already feeds a business workflow, start with one high-volume, reversible action. Map its accepted, refused, incomplete and rejected paths, then run a contract review that includes the downstream owner—not only the prompt owner.
