The eval that belongs in CI, not in a notebook

A platform team at a logistics company had a good eval notebook. When they planned a big change, a model swap or a major prompt rewrite, an engineer opened the notebook, ran a few dozen cases, eyeballed the results, and made a call. It was careful work by careful people. It also missed the regression that actually hurt them.

The regression came in on a small pull request. Someone refactored how the app assembled the context that went into the prompt. The change was three lines, unrelated to the model on its face, and nobody thought to open the eval notebook for it. But those three lines reordered part of the context, and the model started leaning on the wrong field. Answers got subtly worse across a whole category of requests. It went unnoticed for a week, because the only thing that would have caught it was a run nobody ran.

That is the pattern. Manual evals are gated by human judgment about when a change is risky. Humans are decent at that judgment for large changes and blind to it for small ones. The regression does not care how big your diff looked.

An eval a human decides to run will be skipped exactly when it matters

The notebook was not the problem. The notebook was good. The problem was that running it depended on someone deciding, in advance, that a change was model-relevant enough to check. That decision is made from intuition, and intuition says a three-line refactor to context assembly is not a model change. It was.

Anything that only runs when a person remembers to run it will be skipped on the changes that do not look dangerous. And the changes that do not look dangerous are exactly where quiet regressions hide, because nobody is watching them. The fix is to remove the human decision from the loop. If a small eval runs on every pull request automatically, it does not matter whether anyone thought the change was risky. The gate catches it either way.

We did not try to move the whole notebook into CI. That would have been slow and flaky and the team would have turned it off within a month. We picked a small golden set, a few dozen representative cases with known-good expectations, wrote rubric checks that scored the important properties, and set a threshold. That set ran on every PR in a couple of minutes. The full notebook stayed for deep dives before big launches. The two did different jobs.

What good teams do

Keep a small golden set, a few dozen cases, that runs fast enough to sit on every PR.
Gate on properties and thresholds, not exact string match, so normal variation does not turn CI red.
Run it on every change, not only the ones that look model-related, because that is where the quiet regressions live.
Keep the big notebook for deep analysis before major launches. CI catches regressions; the notebook explores.
Make a failure readable: show which case dropped and why, so the person fixing it does not have to reverse-engineer the gate.

Once the golden set was in CI, the next context refactor that would have shifted behavior failed on the pull request itself. The author saw the failing case, understood the reordering was the cause, and fixed it before merge. The whole thing took an afternoon instead of a bad week in production.

How we approach it at Density Labs

Teams usually already have the ingredients for a CI eval. They have a notebook, they have real examples, they have a sense of what good looks like. What they are missing is the small, fast, automatic version that runs without anyone deciding to run it. Our AI Opportunity Assessment is a fixed two-week engagement, priced at $2,500, where we scope the real production work before you build. We look at how you evaluate today, pick the golden set that would have caught your last regression, and design the CI gate around it, sized to run in minutes. You leave with a gate you will actually keep on.

A notebook proves a big change is safe. A CI gate catches the small change nobody thought to check. Keep both, and stop relying on intuition to decide when to look.