The AI incident with no stack trace

A staff engineer at a health tech company described the most frustrating incident of his year. A customer reported that the AI feature had given a dangerous piece of guidance, they had a screenshot, and that screenshot was the entire evidence base. There was no stack trace, because nothing had thrown. There was no error in the logs, because the code had run correctly and returned the model’s answer. The team could not even reproduce it reliably, because feeding the same input back sometimes produced the bad answer and sometimes did not. Every debugging skill the engineer had built over a decade started with an error and a line number, and this incident offered neither. He was debugging a ghost.

Your debugging tools assume a deterministic failure

Debugging, as most engineers practice it, rests on a few assumptions. There is an error. The error has a location, a stack trace that points at the line. The failure is reproducible, so you can trigger it, inspect it, and confirm your fix. These assumptions are so ingrained that they feel like the definition of debugging, and for deterministic code they hold, which is why the tools and the habits work.

An AI incident can violate all three at once. There is often no error, because the failure is a bad answer, not a crash, and a bad answer does not throw. There is no location, because the wrong output did not come from a line of your code, it came from the model, and the model does not have a line number. And it may not reproduce, because the model’s output varies, so the same input can produce a bad answer once and a fine answer the next time. An engineer whose entire toolkit assumes an error, a location, and reproducibility is left with none of them. The instinct to find the throwing line has nothing to find. The instinct to reproduce and bisect breaks on a system that does not repeat itself.

This is why AI incidents feel uniquely maddening, and why they drag. The team is not less skilled. They are applying a debugging model built for deterministic failures to a failure that is behavioral and non-deterministic, and the model gives them nothing to hold. Without the right observability in place beforehand, they are left staring at a screenshot, trying to reconstruct what happened from the one artifact the customer happened to capture.

Build the observability the failure mode needs

The lesson is that AI features need observability designed for their failure mode, captured before the incident, because you cannot reconstruct a non-reproducible bad answer after the fact from logs that only recorded that nothing errored. The trace you need is not a stack trace. It is a record of what the feature actually did.

What teams capture so an AI incident has something to grab.

The full input, context, and prompt that produced each output, so you can see exactly what the model was asked, not just that it was asked.
The actual output the feature returned, stored and searchable, so a reported bad answer can be found and examined.
The model version and configuration in effect at the time, so you can tell whether something changed underneath you.
Enough context to attempt a replay, knowing it may not reproduce, and to reason about it when it does not.
Quality signals over time, so a rise in bad answers is visible as a trend even when no single one throws.

The point is to make the invisible failure visible, ahead of time, because after the fact there is no stack trace to recover. The observability is the stack trace, and it only exists if you built it before you needed it.

How we approach it at Density Labs

The AI incident with no stack trace is a common and demoralizing experience for strong teams, and it is not a skills gap. It is a tooling gap. The team’s debugging toolkit assumes an error, a location, and reproducibility, an AI failure often has none of those, and without observability built for the failure mode, there is genuinely nothing to debug with. The screenshot is all they have because it is all anyone captured.

Our AI Opportunity Assessment is a fixed two-week engagement, priced at $2,500, that scopes the real work of shipping an AI feature. Part of that is the observability, capturing inputs, outputs, versions, and quality signals, so when a bad answer gets reported, the team has a record to investigate instead of a ghost to chase.

Your debugger looks for a throwing line. An AI failure does not throw. You have to record what it did before it does it.