When the honest scope is automate most, escalate the rest

An operations lead at a claims-processing outsourcer told me his AI project spent four months trying to do something it should never have promised. The goal, as written, was to automate claims intake. All of it. Every claim that came in would be read, categorized, and processed by the model, no human in the loop. That was the pitch that got funded, and it was the pitch that stalled.

The problem was the tail. Most claims were ordinary. Same structure, clean data, obvious category. The model handled those easily. Then there were the strange ones. A claim with a handwritten addendum. A claim that referenced three prior claims. A claim where the numbers did not add up and someone had to decide whether that was fraud or a typo. Those were rare, but they were exactly the cases where a wrong automated decision cost real money, and the team kept pouring effort into teaching the model to handle them.

Every week of that effort bought a little more coverage of the tail and a lot more fragility everywhere else. He finally reset the scope. The model handles the common cases end to end. Anything it is not confident about, or anything that matches a short list of tricky patterns, goes straight to a human. That version shipped in weeks and worked. The all-or-nothing version never would have.

Full automation is often the wrong target

The instinct to automate everything comes from a good place. Partial automation feels like failure, like you did not finish the job. In practice, the last slice of hard cases is where most of the cost and most of the risk live, and chasing it can sink the whole project.

The math is quietly brutal. Handling the common cases might be a few weeks of work. Handling the rare, weird, high-stakes cases might be most of the remaining effort, and it is the effort most likely to produce a confident wrong answer on exactly the claim you could least afford to get wrong. You spend the majority of the budget buying the riskiest coverage.

A head of billing at a utility company described the same shape. Her team wanted AI to resolve every customer dispute automatically. The version that worked resolved the routine disputes and escalated anything involving a disputed amount over a threshold, or anything with an angry tone, to a person. The escalation path was not a weakness in the design. It was the design.

Scope the split, not the total

The honest scope names two things: what the model owns, and what it hands off. Getting the handoff right is most of the work.

Decide what “common” means concretely, so the model knows what it is allowed to finish on its own.
List the patterns that must escalate, the handwritten note, the multi-claim reference, the numbers that do not reconcile.
Give the model a way to say “I am not sure” and route on low confidence, not just on known patterns.
Design the human queue as a real part of the system, staffed and measured, not a leftover.

Scoped this way, the model does the volume, the people do the judgment, and neither is asked to do the other’s job. Coverage of the common cases is where the value is, and it is also where the model is safe. The tail is where you keep the human, on purpose.

The outsourcer’s reset was not a retreat. It was the first honest statement of what the system should do, and it was faster to build because it stopped pretending the tail was easy.

How we approach it at Density Labs

In the AI Opportunity Assessment, our two-week fixed engagement at $2,500, we look for the common-case majority and the risky tail early, because the split usually decides whether a project ships in weeks or stalls for quarters. We ask what the model can safely own, what has to reach a person, and how the handoff works when it is not confident. Naming that boundary up front is cheaper than discovering it four months into a build that promised more than it should have.

Automating the easy majority and escalating the hard minority is not a compromise. It is usually the right answer written down honestly.