Graceful degradation: what your AI feature does when the model is unavailable

A product engineer at a media company showed me what their AI feature did when the model was unavailable. The feature suggested tags for articles as editors wrote them. When the model was up, it was genuinely useful. When the model was down, the whole editor threw an error and would not let anyone save their work.

An optional convenience had become a hard dependency. A helper that was supposed to make one task easier had quietly taken the entire workflow hostage. The model went down for a while, and editors could not do their core job, which had nothing to do with tags.

Nobody decided that on purpose. It was just what happened, because nobody had designed the state where the model is gone.

The lesson: design the degraded state as a first-class product state

Every AI feature has a state where the model is unavailable. It will be slow, rate limited, erroring, or down. The only question is whether you designed what happens in that state or let it be an accident. When it is an accident, it is almost always the worst version: a hard failure that takes more with it than the AI part.

So design the degraded state deliberately, as a real product state with its own defined behavior. The right fallback depends on the feature:

A cached or previous answer, when a slightly stale result beats no result.
A simpler heuristic or rule, when a rough answer is better than nothing.
A human path, when the safe move is to route to a person.
A clean, honest unavailable state that still lets the user do everything the AI was only assisting.

For the media company, the fix was small once they named the problem. When the model is unavailable, the tag suggestions simply do not appear, and editors write and save exactly as they would without the feature. The AI is a bonus when present and invisible when absent. It never blocks the core task again.

What good teams do

Name the states where the model is unavailable, and design behavior for each.
Keep the AI assistive where it should be assistive, so its absence never blocks the core task.
Pick a fallback that fits the feature: cache, heuristic, human, or a clean unavailable state.
Make the degraded state honest to the user instead of a confusing half-broken screen.
Test the feature with the model turned off, as a normal case, not an afterthought.

That last point is where most teams fall short. They test the feature with the model working and never once with it off. The degraded path ships untested, and its first real run is a live incident. Turning the model off in testing, on purpose, is how you find out whether your fallback is real.

How we approach it at Density Labs

Hard failures on model unavailability are one of the most avoidable AI production problems, and one of the most common. The feature works beautifully in the demo, where the model is always up, and nobody designs the state where it is not. That is an operations and delivery gap, the same place most enterprise AI efforts stall rather than on model quality.

Our AI Opportunity Assessment is a fixed two-week engagement for 2,500 dollars that scopes the real production work before you build. For any AI feature going into a real workflow, part of that scope is defining the degraded state, choosing the right fallback, and confirming the feature has been tested with the model off, so an outage means less magic, not a broken product.

The model will be unavailable sometimes. Decide now what your feature does, before your users find out for you.