Cut the ten percent that doubles the cost
An engineering director at a streaming service almost killed a good AI project over the rarest slice of its own use case. The fix was to cut that slice, not to solve it.
Cut the ten percent that doubles the cost
An engineering director at a streaming service told me about a project that stalled for two months over a problem almost nobody would ever hit. The team wanted an AI system to answer subscriber questions about billing, plans, and account changes. Most of those questions were ordinary. What plan am I on. Why was I charged twice. How do I cancel. The system handled them cleanly in early tests.
Then someone raised the hard cases. Customers on legacy grandfathered plans that no longer existed in the pricing system. Accounts split across a family that had merged and unmerged over years. Refunds tangled up with promotional credits and partial months. Real situations, all of them. And every one of them needed special logic, special data access, and a human to confirm the answer before it went out.
The team did what careful teams do. They started building for the hard cases. That is where the two months went.
The tail is where the budget dies
Here is the shape the director eventually saw, and it is common. The easy ninety percent of a use case is cheap. The rare, messy ten percent is not a little more expensive. It often costs as much as everything else combined, sometimes more. It needs edge-case handling, extra integrations, review steps, and a longer tail of testing because you cannot reproduce the situations easily. You spend the back half of your budget serving the front edge of your problem.
The trap is that the hard cases feel important. They are important, to the customers who have them. But importance to a few people is not the same as value to the project. When you scope the whole thing as one system, the tail sets the timeline and the cost for the entire build, and the ordinary cases wait behind it.
The director’s move was simple once he named it. Draw a line. The AI answers the ordinary billing questions on its own. Anything touching a legacy plan, a merged account, or a mixed refund gets routed straight to a human, untouched. Not a worse product. A shipped one. The rare cases were already handled by people before the project started, and those people kept handling them.
A colleague of mine who leads data work at an insurer described the same lesson from the underwriting side. Her team could automate standard policies fast. The exotic risks, the ones that show up a handful of times a year, would have doubled the model’s complexity for a sliver of the volume. They shipped the standard path and left the exotic ones with the senior underwriters, who were better at them anyway.
Scope the cut on purpose, early
The reason to do this in discovery, not in month three, is that the cut changes everything downstream. If you decide up front that the AI owns the common cases and humans own the tail, you design a clean handoff from day one. You size the model for the volume it will actually serve. You write the acceptance tests against the cases you are keeping. You stop paying to solve problems you were never going to let the AI touch.
If you wait, the tail quietly expands the project until the easy part is late too. That is how a good idea becomes a stalled one.
A few questions worth asking before you commit to serving the whole use case:
- What fraction of the volume lives in the hardest cases?
- What does each hard case add in integrations, review steps, and test surface?
- Who handles those cases today, and are they worse at it than an AI would be?
- Can you draw a clean line that routes the tail to a human without confusing the user?
If the rare slice is doubling your cost and it is already handled well by a person, that is not a case to solve. It is a case to route.
How we handle this at Density Labs
Scoping the cut is one of the first things we do in the AI Opportunity Assessment, our two-week fixed engagement at $2,500. We map the real distribution of a use case, find where the cost concentrates, and draw the line between what the AI should own and what should stay with people. Often the most useful thing we deliver is permission to leave the hardest ten percent alone, because that is what lets the other ninety ship this quarter instead of next year.
Do not let the rarest, hardest slice of a use case set the price of the whole thing. Cut it, route it, and ship what is left.