Schema-constrained output is a contract, not a suggestion

A staff engineer at a supply-chain company told me about a break that made no sense at first. His feature had run cleanly for a long stretch. Then one afternoon a downstream service started throwing errors, and the input that triggered it was an ordinary shipment record. Nothing about it looked special.

The model in the middle produced a structured object that the downstream service consumed. Most of the time the object had a nested list under one key. On this record, the model returned that key as a single value instead of a list. The downstream code, which had only ever seen a list, tried to iterate and fell over. The data was reasonable. The shape had shifted.

“Usually the right shape” is a trap

The whole system rested on a quiet assumption. The model usually returned the shape the downstream service expected, so the team treated “usually” as “always.” They had built the integration on top of a pattern the model tended to follow, and the tending was good enough for a long time, which is exactly what makes it dangerous. A shape that holds for months feels like a guarantee. It is not one.

A prompt that describes a schema is asking the model to please produce that shape. The model complies most of the time because the request is clear and the examples point that way. But a language model chooses its output token by token, and on an unusual input it can choose a slightly different structure that still reads as a valid answer. A key that is usually a list comes back as a scalar. An object that usually has five fields comes back with four. Each of these is a small, local decision by the model, and each one is a break at the integration seam that assumed the shape was fixed.

Enforce the schema, do not hope for it

The fix is to stop treating the schema as a hope and start treating it as a contract that is enforced on both sides. Enforcement means two things: constrain generation so the model can only produce the valid shape, and validate the result before anything downstream touches it.

Use constrained decoding where you can. Many models support a mode that forces output to conform to a schema. When the model is physically prevented from emitting the wrong shape, the rare-input break cannot happen at generation.
Validate against the schema anyway. Parse the output and check it against required fields, types, and structure before it leaves your boundary. A list that should be a list is verified as one, not assumed.
Reject, do not coerce silently. If the shape is wrong, treat it as an error you catch and handle, with a retry or a fallback. Do not quietly reshape it and pass it along, because that hides the failure.
Test the rare inputs on purpose. The break lives in the unusual record. Feed the feature edge cases and odd shapes and confirm the contract holds, rather than waiting for a real one to find it.

The principle is that any boundary between two systems needs an enforced contract, and a model sitting between them is no exception. The model’s tendency to produce a shape is not the same as a promise to produce it. The promise has to come from constrained generation and a validation gate, not from the model’s good behavior on the inputs you happened to test.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week, $2,500 engagement that scopes real production work before you build, we look at every place a model’s output feeds another system and ask what happens on the shape it produces rarely rather than usually. That is where these breaks hide. We put the schema under constrained decoding where the model supports it, and we add a validation gate that rejects a wrong shape instead of passing it downstream. Finding the rare-shape break in a design review is cheap. Finding it when a real shipment record takes down the integration is not.

A shape the model usually returns is a suggestion. Enforce it, and it becomes a contract.