The second model you added to check the first is not free

A staff engineer at a document-processing company walked me through a guardrail he was pleased with. The main model extracted data from a document; a second model call reviewed the extraction and flagged anything wrong. It caught real mistakes. It also, he admitted when I asked, doubled the cost of every request and added a noticeable wait, and he had never checked how often the reviewer itself was wrong. The technique was sound. He had adopted it as if it were free, and it was not.

A reviewer model is a second bill and a second fallible judge

Adding a model to check a model is a legitimate pattern. It can catch failures a deterministic rule would miss, especially for fuzzy judgments about tone, relevance, or whether an answer actually addresses the question. But every one of those checks is a full model call, so you are paying twice per request and waiting twice, and at volume that is a real line on the bill and a real hit to latency.

The part the engineer had skipped is that the reviewer is also a model, with the same nondeterminism and the same capacity to be confidently wrong. It can approve a bad output. It can reject a good one. Stacking a second fallible judge on the first reduces some errors and introduces its own, and if you never measure the reviewer, you do not actually know whether it is helping or just costing you. A guardrail you have not measured is a belief, not a control.

So the reviewer was doing useful work and carrying hidden costs at the same time, and the team had booked only the useful half.

Use it deliberately, and prefer the cheap check where it fits

The move is to treat a reviewer model as a deliberate choice with a price, not a default you reach for whenever you want more safety. Before you add one, ask whether a deterministic check would catch the same failures for almost nothing. A lot of what teams reach for a second model to verify is checkable in plain code: is the output valid JSON, are the required fields present, is the number in range, does the value exist in the source document. Those checks are instant, free, and never wrong in the way a model is.

The document team found that much of what their reviewer caught was structural, exactly the kind of thing a schema check handles. Once the cheap checks took that load, the reviewer model was left doing only the genuinely fuzzy judgments, which is where it earned its cost.

Here is how to decide.

Try a deterministic check first. Validate format, ranges, required fields, and source-grounding in code, because these are free, fast, and reliable, and they cover a lot of ground.
Reserve the reviewer model for fuzzy judgment. Use it where correctness depends on meaning, tone, or relevance that plain code cannot assess.
Measure the reviewer. Track how often it approves bad output and rejects good output, so you know whether it is improving quality or just adding cost.
Count the cost and latency. Know what the extra call adds per request and at scale, and decide whether the failures it catches are worth that.
Consider a cheaper reviewer. A smaller, faster model may verify well enough for a fraction of the price, so the check does not cost as much as the work it is checking.

Used this way, the second model is a tool you point at the problems only it can solve, while the cheap checks handle everything else.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week, $2,500 engagement, we look hard at guardrails that are themselves model calls, because they are easy to add and easy to leave unmeasured. We separate the checks a deterministic rule can do for free from the ones that genuinely need a model’s judgment, put the cheap checks first, and measure whatever reviewer remains so its cost is a decision rather than a surprise. It is far cheaper to right-size that before launch than to find the doubled bill and the wait in production.

Checking a model with another model can be worth it. Just remember you are paying twice for a judge who can also be wrong.