The kill switch every AI feature needs before its first user
When it goes wrong, the fastest safe move is off. Make sure you can.
The kill switch every AI feature needs before its first user
A VP of engineering at a consumer app told me about the night an AI feature started misbehaving in a way that was clearly worse than having no feature at all. The team agreed quickly on what to do. Turn it off. Fall back to the old behavior. Investigate calmly in the morning.
Then they realized they could not turn it off.
The feature was woven straight into the request path. There was no flag, no toggle, no switch. Disabling it meant a code change, a review, a build, and a deploy, in the middle of the night, under pressure, with customers hitting the broken behavior the whole time. What should have been a ten-second decision became an hour of rushed engineering, which is exactly the condition under which people introduce a second bug.
The lesson: ship the off switch before the feature
The fastest safe action when an AI feature goes wrong is almost always to stop using it and fall back to whatever the product did before. That is only possible if two things are true. There has to be a switch that disables the feature without a deploy. And there has to be a defined behavior to fall back to.
Both need to exist before the first user ever touches the feature, because the moment you need them is the worst moment to build them.
- Put the feature behind a flag that can be flipped off instantly, without a code deploy.
- Define the fallback explicitly: the previous behavior, a simpler path, or a clear unavailable state.
- Make sure flipping the switch actually routes to that fallback, and test that path before launch.
- Give the switch to the people who will be awake during an incident, not just the original authors.
- Confirm the off state is safe and coherent, not a half-disabled feature that breaks in a new way.
The consumer app added a flag the week after that night. The next time something looked wrong, an on-call engineer flipped it, the product returned to its known-good behavior, and the investigation happened the next day with no customers in the blast radius.
What good teams do
Good teams treat the kill switch as part of the feature’s definition of done, alongside the code and the tests. The feature is not ready to ship because it works. It is ready to ship because it works and because you can turn it off cleanly when it does not.
They also rehearse the off switch. A flag that has never been flipped is a flag you do not fully trust. Toggling it once in a controlled way, before launch, confirms the fallback path is real and that the disabled state behaves. An untested kill switch is a hope, not a control.
How we approach it at Density Labs
Missing kill switches are one of the clearest signs that an AI feature was built as a demo and shipped as a product. The model works in the happy path, and nobody planned for the moment it does not. That gap is squarely in operations and delivery, which is where most enterprise AI efforts stall rather than on model quality.
Our AI Opportunity Assessment is a fixed two-week engagement for 2,500 dollars that scopes the real production work before you build. For any AI feature headed for real users, part of that scope is confirming there is a real off switch that works without a deploy, a defined fallback behind it, and a tested path to that fallback. We name who can flip it and how fast.
Build the off switch first. You will want it before your users do.