Your runbook assumes failures that an AI feature does not have

A tech lead at a logistics company showed me the runbook their on-call team had used during an AI feature incident, and why it had been useless. The runbook was a good one, for a normal service. Check the error logs. Restart the service. Roll back the last deploy. Verify the dependencies are healthy. The on-call engineer had done all of it, correctly, and none of it helped, because the feature was not failing in any of the ways the runbook assumed. There were no errors in the logs. Restarting did nothing, because the service was not stuck. Rolling back the deploy did nothing, because the last deploy was not the cause. The runbook was written for failures with a clear mechanical cause, and this failure did not have one.

Runbooks encode a model of how things fail

A runbook is a compressed theory of how a system breaks. Each step exists because, historically, that check or that action resolved a real class of failure. Check the logs, because failures usually leave errors. Restart, because processes get stuck. Roll back, because the last change is a common culprit. The runbook works when the failures match the theory baked into it, and for deterministic services they usually do, because those services fail in the ways the runbook expects.

AI features fail outside that theory. The common failure is not an error, it is a wrong answer, which leaves no trace in the error logs because nothing errored. Restarting does not help, because the feature is not stuck, it is confidently producing bad output. Rolling back the last deploy does not help when the cause is a model provider’s update, or a shift in the input data, or a prompt change from a week ago that only started biting now. The mechanical steps that resolve deterministic failures have nothing to grab onto, because the failure is not mechanical. Following the runbook to the letter produces no progress, and it produces something worse than no progress, a false sense that the right things are being tried while the incident continues.

The deeper issue is that the runbook does not even point at the questions that matter for an AI feature. It does not tell you to check whether the output quality dropped, whether the model changed underneath you, whether the inputs shifted, or whether a recent prompt edit is the cause. Those are the real levers, and the runbook says nothing about them, because it was written before anyone knew this feature could fail this way.

Rewrite the runbook around how AI actually fails

The lesson is that an AI feature needs a runbook written from its own failure modes, not a copy of the template that works for deterministic services. The steps have to point at the causes that are actually plausible for a feature that fails by giving wrong answers.

What an AI feature’s runbook covers that a standard one does not.

Check output quality directly, with the signals and samples that reveal a bad-answer failure the error logs will never show.
Check whether the model changed, since a provider update can shift behavior without any deploy on your side.
Check whether the input data shifted, because a change in what the feature is fed can break it while the code is untouched.
Review recent prompt and configuration changes, not just code deploys, since those change behavior and often are not in the deploy history.
Give a way to fall back to a safe mode or a previous prompt, since the fix is often reverting a behavior change, not restarting a process.

The old runbook steps are not wrong, they are just aimed at a kind of failure this feature rarely has. The runbook has to aim at the failures the feature actually produces.

How we think about it at Density Labs

The useless runbook is a common find during AI incidents, and it is not because the runbook was bad. It was a fine runbook for a deterministic service, applied to a feature that fails in ways that runbook never imagined. On-call did everything right and got nowhere, because the steps were built around mechanical failures and the feature had a behavioral one.

When we help a team make an AI feature operable, we write the runbook from the feature’s real failure modes, so on-call is pointed at output quality, model changes, input shifts, and prompt edits, the things that actually cause an AI feature to fail. It turns a page that stalls into a page someone can resolve.

A runbook is a theory of how your system fails. If it was written for a service that errors, it will not help with a feature that lies.