The on-call question every AI feature has to answer before launch

A reliability-minded engineering lead at a payments company told me about the double standard he kept seeing. His org had rigorous on-call for every traditional service. Someone was always paged, always accountable, always ready. Then the company started shipping AI features, and those went out with none of it. When an AI feature produced a bad output at a bad time, there was no one on call for it, no clear owner to page, no defined response. The first real incident became a scramble to figure out who was even responsible, while the problem kept happening. The AI features had skipped the operational maturity every other service was held to.

He asked the question that should be asked before any launch. When this misbehaves at two in the morning, who gets paged, and what are they expected to do? For the traditional services, the answer was instant. For the AI features, it was a shrug, and a shrug is not an incident response.

AI features are services, and services need someone on call

An AI feature in production is an operational commitment, the same as any service that can fail. It just fails in less familiar ways, which makes the on-call question more important, not less. A wrong output at scale, a cost spike, a sudden drift in quality: these are incidents, and an incident with no owner is an incident that runs unchecked until someone happens to notice. Skipping the on-call design just moves the cost to the first time the feature breaks, multiplied by the time it takes to find a responder.

The teams that treat AI features as real services answer the on-call question before launch. There is a named owner in the escalation path. There is a defined response for the failure modes that are specific to AI, not just the ones borrowed from traditional monitoring. And the person who gets paged knows what to do when the failure is a bad answer rather than a crashed process.

Settling on-call before you ship

Name who gets paged. A specific person or rotation accountable for the live feature, decided before launch, not discovered during the incident.
Define the AI-specific failures. Wrong output, cost spike, quality drift. These need a response, not just server metrics that all look green while the feature misleads people.
Give the responder a playbook. What to do when the failure is a bad answer. Improvising the response at two in the morning is how small incidents become big ones.
Hold AI to the same bar as any service. If a traditional feature would not ship without on-call, an AI feature should not either. The novelty is not an excuse.

How we approach it at Density Labs

In our AI Opportunity Assessment ($2,500), we answer the on-call question before anything ships. We name who gets paged, define the AI-specific failure modes that traditional monitoring misses, and make sure there is a real response, not a scramble. Deciding this on paper costs a short conversation. Deciding it live, during the first incident, costs the incident and the trust that goes with it.

An AI feature is a service that can fail in unfamiliar ways. Ship it with someone on call, or plan to meet its owner for the first time during the outage.