# Red flags that your AI use case won't survive production

_Some pilots demo beautifully and are already doomed. The warning signs are visible at scoping, if you know which ones to look for._

The red flags at scoping time that tell you an AI use case won't survive production, from untrustworthy data to no single source of truth. An anonymized story from an engineer-turned-founder.

# Red flags that your AI use case won't survive production

A pilot can look healthy right up until the moment it has to run for real. The demo is clean, the stakeholders are happy, and the thing quietly cannot survive production. The frustrating part is that the warning signs were usually visible back at scoping. You just have to know which ones to take seriously.

The most reliable red flag hides in the data. Pilot data is almost always a clean, curated subset. Production data is not. A use case that only works on the tidy sample, and has never been tested against the mess, is carrying a failure it has not met yet.

## The red flags an engineer learns to smell

An engineer-turned-founder who now runs an equity-management startup put words to something I see constantly. His principle is that AI is knowledge and humans are intelligence, so you should never hand the AI your judgment. The way you keep judgment in the loop, he argued, is to take the hundreds of rules and decisions that live in your head and codify them into an explicit system that steers the AI, instead of hoping a giant prompt gets it right. A use case with no such rules written down is a red flag. It means the standard for "correct" lives in someone's intuition and cannot be enforced at scale.

He was just as clear about data. In his world, when ownership records live scattered across spreadsheets, lawyers, and accountants with no single source of truth, founders make decisions on unreliable data, and it comes back to bite them during due diligence for a raise. Read that as a production warning. If your AI use case sits on data that has no single trustworthy source, the pilot will look fine on the curated slice and fall apart the moment it runs against the real, contradictory records. Messy governance is not a paperwork problem. It is a production time bomb.

What makes these red flags dangerous is that a pilot actively hides them. During scoping, someone hands you a clean export to test against, precisely because it is easy to work with. The messy, contradictory, half-governed reality never enters the pilot, so the pilot never shows the failure. You get a green light that was measured on the one version of the data that does not exist in production. The flag was there the whole time. The pilot was just built to look past it.

## The red flags to check before you commit

Run the candidate use case past these, and treat a yes as a stop sign.

- **No single source of truth.** The data lives in several places that disagree.
- **Correctness lives in someone's head.** There are no written rules the output can be checked against.
- **Only tested on the clean subset.** Nobody has run it against the messy production data.
- **No plan for being wrong.** No monitoring, no owner paged, no path to catch a bad output.

Any one of these means the use case can pass a demo and still not survive a week of real traffic.

## How we approach it at Density Labs

Spotting these before you build is exactly what the AI Readiness Assessment, our $2,500 engagement, is for. We look at where the data actually lives, whether there is a single trustworthy source, whether the rules for "correct" are written down or only felt, and what happens the first time the output is wrong. When we find these red flags, we say so plainly, because it is far cheaper to fix a data source or codify a rule now than to discover it in production.

A clean demo proves the use case can work on the easy data. Production asks whether it survives the real data. Check that before you build, not after.