Feature flags for AI: changing the model without shipping a deploy

An engineering manager at a logistics company told me about a model upgrade they wanted to do carefully and ended up doing recklessly, because their setup gave them no other option. A new model looked better in testing, and they wanted to roll it out to ten percent of traffic, watch the quality, and expand if it held. But the model was hard-coded, so switching it meant a code change and a full deploy, and there was no mechanism to send it to only some traffic. Their only real option was to swap the model for everyone at once and hope. They deployed it to all users, the new model was worse on a segment their tests had not covered, and they had to scramble a rollback for a change they had wanted to make gradually all along.

AI changes are exactly the kind you want to roll out gradually

Feature flags exist because some changes are risky enough that you do not want to turn them on for everyone at once. You flag them, roll them out to a fraction of traffic, watch, and expand or retreat based on what you see. This is standard practice for risky product changes, and it works because it turns a binary bet into a controlled experiment.

Model and prompt changes are among the most flag-worthy changes a team can make, and they are often the least flagged. A new model can behave differently in ways your testing did not catch, because the input space is huge and your tests cover a slice of it. A prompt change can improve most cases and quietly break a segment. These are precisely the changes where you want to roll out gradually, compare the new behavior against the old on real traffic, and catch the regression on ten percent instead of a hundred. Yet teams routinely hard-code the model and the prompt, which turns every change into an all-or-nothing deploy, the exact pattern feature flags were invented to avoid. The change that most needs a gradual rollout gets the least gradual mechanism.

The result is that teams either avoid changing the model, which leaves them stuck on an aging version, or they change it for everyone and absorb the occasional bad surprise across their whole user base. Both are worse than the flagged rollout that the situation is begging for, and both come from the same gap, the AI behavior was wired as a constant instead of as something you can vary safely.

Put the model and prompt behind flags

The lesson is that model and prompt changes should be controlled the way any risky change is, behind a flag, rolled out gradually, comparable against the current behavior. The behavior-defining parts of an AI feature deserve the rollout tooling you already use for risky product changes.

What teams do to flag AI behavior.

Put the model choice behind a flag, so you can move a fraction of traffic to a new model without a deploy.
Put prompts behind configuration you can vary by segment, so a prompt change can roll out gradually instead of all at once.
Run new and current behavior side by side on real traffic, so you can compare quality before committing everyone.
Expand or retreat based on the live quality signal, catching a regression at ten percent rather than a hundred.
Make the switch instant and deploy-free, so rolling back a bad model change is fast when it needs to be.

None of this is exotic. It is the feature-flag discipline teams already have, applied to the layers of an AI feature that most need it and usually do not get it.

How we think about it at Density Labs

The hard-coded model is a common and quietly expensive pattern. It leaves the team with two bad options, stay frozen on an old version, or flip the whole user base to a new one and gamble. Both come from wiring the model and prompt as constants, when they are the parts of the feature that most need gradual, reversible rollout.

When we help a team build an AI feature, we put the model and prompt behind flags from the start, so behavior changes can be tried on a slice of traffic, measured, and expanded or pulled back. It turns model upgrades from a gamble into a routine, controlled operation.

The model and the prompt are the riskiest things you will change about an AI feature. They should be the most flagged, not the most hard-coded.