The model is acting weird, and what that becomes at 3 a.m.

An engineering manager at a fintech company read me a message from their incident channel, sent at three in the morning. It said, in full, “the model is acting weird.” That was the alert. The on-call engineer who received it had no idea what weird meant, which model, which feature, how weird, or what to do about it. They spent the first hour of the incident just trying to figure out what the incident was. By the time they understood the actual problem, output quality had dropped for a specific segment of users, real time had been lost to translating a vague complaint into a concrete failure that could be acted on. The alert had a feeling in it. It had no information.

Vague signals cost the most when it is late and quiet

Alerting for normal services tends to be specific because the failures are specific. Latency crossed a threshold. Error rate exceeded a bound. A disk filled. The alert names the thing that is wrong and often points at where, which is why an on-call engineer can act on it quickly even half asleep. The specificity is what makes the alert useful.

AI features attract vague alerts, because the failure is fuzzy and the people reporting it do not have a precise vocabulary for it. The model is acting weird. The answers seem off. Something feels wrong with the assistant. These are real observations, but they are not incident signals, they are impressions, and an impression is nearly impossible to act on at three in the morning. The on-call engineer cannot check “weird” on a dashboard. They have to first reconstruct what weird means, which segment, which inputs, how bad, before they can even begin to respond, and every minute of that reconstruction is a minute the failure continues. The vagueness is not a small annoyance. It is the difference between a five-minute response and a two-hour one.

The root problem is that the team never defined what a quality failure looks like precisely enough to alert on it. So the only signal that exists is a human noticing something feels off, which is late, imprecise, and dependent on someone being awake to feel it. There is no crisp threshold that fires, because nobody built one, because building one requires deciding what bad output actually is.

Turn the feeling into a signal

The lesson is that on-call for an AI feature needs signals as specific as any other service’s, which means defining what failure looks like concretely enough to measure and alert on. The model is acting weird has to become a metric that crosses a threshold, or it will keep arriving as a shrug at the worst possible time.

What teams alert on instead of “acting weird.”

A measurable quality signal, such as a graded score on live samples, that fires when it drops below a defined bar.
Specific failure rates, like the rate of the feature refusing valid requests or producing outputs that fail a format check.
Downstream proxies, such as a spike in the human-correction rate or in customers retrying, that reveal quality problems in numbers.
The model version changing, so a provider update pages someone with a specific cause instead of a vague feeling.
Segment-level signals, so a failure hitting one group of users is named as such and not lost in a healthy average.

The aim is that when something goes wrong with the feature, the alert says what, where, and how bad, so on-call can act in minutes instead of spending the first hour deciding what the incident even is.

How we think about it at Density Labs

The vague AI alert is a symptom of a gap upstream. The team never defined output quality precisely enough to measure it, so the only available signal is a person noticing something feels off and typing it into a channel. That signal is late, imprecise, and useless at three in the morning, and it is what most teams are running on because defining the quality signal is work nobody scheduled.

When we help a team make an AI feature operable, we turn the fuzzy failure into concrete signals with thresholds, so on-call gets an alert they can act on instead of a feeling they have to decode. The work is in deciding what bad output means. Once that is defined, the alerting follows.

The model is acting weird is not an incident report. It is a request for someone else to figure out what the incident is, and that someone is asleep.