What the pilots that shipped had in common

The failure numbers get all the attention. MIT put the share of enterprise AI pilots that return nothing at about 95%. The more useful question runs the other way. What did the 5% do? I have watched enough of both to notice that the ones that shipped were not smarter about models. They were disciplined about a few small things that the stalled pilots skipped.

They started narrow and refused to widen

A CTO at a fintech told me about a feature that made it to production while three flashier ones died. The shipped one did a single job. It drafted a first-pass response to a specific class of support ticket, nothing else. He kept turning down requests to make it also route tickets, also tag them, also summarize the thread. Every one of those would have been reasonable. Together they would have turned a two-month build into a two-year one.

The stalled pilots at the same company had the opposite shape. They tried to be useful everywhere, so they were finishable nowhere. McKinsey’s 2025 work suggests roughly 70% of enterprise AI use cases are served well enough by an off-the-shelf option, which means the room to build something custom is narrow to begin with. The teams that shipped picked one slice inside that narrow room and defended it.

They wrote down what “working” meant before they built

A director of engineering at a messaging startup gave me the other trait. Before her team wrote a line of the feature, they wrote the test. Not unit tests. A set of about forty real inputs with the outcome they expected for each. When someone claimed the feature was ready, the claim was checkable in an afternoon instead of being a matter of opinion.

She said the baseline mattered more than the model. They measured how the manual process performed first, then held the AI feature to that number. A pilot without that baseline cannot prove it worked, which is why so many of them quietly get shelved. The team knew their bar before they started, so they knew the day they cleared it.

The shipped pilots, across very different companies, kept showing the same handful of habits:

A single job, scoped so tightly that “done” was obvious to everyone.
A written baseline, captured before deployment so the ROI claim could be falsified.
A real test set, a few dozen actual inputs with expected outcomes, run against every change.
A named owner, one person accountable for the feature reaching and staying in production.

None of these is about the frontier of AI. All of them are about closing the gap between a demo and a system somebody depends on.

How we approach it at Density Labs

When we run an AI Opportunity Assessment, a fixed two-week engagement priced at $2,500, we are mostly looking for whether these habits are present or reachable. Is the use case narrow enough to finish? Does a baseline exist, or can we capture one in the two weeks? Are there real inputs to build a test set from? Is there a person willing to own the result? Those questions predict production far better than any model score. The engagement exists so you learn the answers before you commit the build budget, not after.

The pilots that shipped were not lucky. They were narrow, measured, and owned, and each of those was a decision somebody made in the first week.