Run the manual baseline before you scope the AI feature

A finance operations lead at a subscription business was about to scope an AI feature for matching incoming payments to open invoices. It was a real pain point. Payments came in through several channels, invoices lived in the billing system, and someone spent hours each week reconciling the two. An AI matcher sounded like an obvious win.

Before writing the scope, she did something most teams skip. She sat down and did the matching by hand for a week, on a small batch, paying attention to how she actually made each decision. Not to feel the pain. To learn the rules.

That week changed what she asked for, and it probably saved the project.

Doing it by hand teaches you the real rules

When you scope an AI feature from the outside, you assume the task is simple because you have never done it carefully. Matching a payment to an invoice sounds like comparing an amount and a reference number. Easy.

Doing it by hand taught her it was nothing like that. Customers paid several invoices with one lump sum. Customers paid the wrong amount and expected you to figure out which invoice they meant. Reference numbers were often missing, wrong, or a customer’s own internal code that meant nothing to her system. She was not comparing fields. She was using context, judgment, and sometimes a guess. Half the rules she followed were things she had never articulated until she watched herself do them.

Those unwritten rules are the actual specification for the AI. You cannot get them from a stakeholder interview, because the person doing the work does not know they know them. You get them by doing the work and noticing what you do. Every time she paused and thought “well, in this case I check the customer’s history,” that was a rule the AI would need. Every “this one I just flag for a human because it is ambiguous” was a boundary the AI would need too.

The baseline sets the bar the AI has to clear

The second thing the manual week gave her was a number that mattered more than any benchmark. She saw how often a careful human got it wrong. Sloppiness had nothing to do with it. Some matches are genuinely ambiguous, and any human, or any model, will sometimes guess wrong.

That human error rate is the bar. AI features generally need to reach about 85 percent accuracy before their mistakes stop compounding into something worse than the manual process, but the real target is set by how well a person does the same task. If a careful human matches correctly most of the time and flags the truly ambiguous cases for review, then the AI’s job is not to be perfect. It is to match at least as well as the human on the clear cases and, just as importantly, to flag the ambiguous ones instead of guessing confidently. She would never have known to require that second behavior if she had not felt, by hand, how often the honest answer was “I am not sure, someone should look.”

A data lead at an insurance carrier told me she learned the same lesson doing a batch of underwriting checks manually before scoping a tool. The value of the exercise was not the checks. It was discovering that a good human said “refer this to a senior” surprisingly often, and that a model which never said that would be worse than the person even at higher raw accuracy.

Small scale is enough

You do not need to do the whole job by hand. You need enough of it to hit the weird cases and to watch your own decisions. A week, a hundred examples, whatever surfaces the messy minority. The goal is to leave with three things. The unwritten rules the task actually follows. The rate at which a careful human gets it right. And a clear sense of which cases a human refuses to decide alone, because the AI should refuse those too.

Scope written after that week looks different from scope written before. It is narrower where the task is genuinely ambiguous, it names the flag-for-review behavior as a requirement, and it sets an accuracy bar tied to real human performance instead of a number someone liked the sound of.

How we approach it at Density Labs

In the AI Opportunity Assessment, our two-week fixed engagement at $2,500, we do the task by hand before we scope the automation. We work a real batch, write down the rules we find ourselves using, and measure how a careful person actually performs, including how often the right move is to refer rather than decide. That baseline becomes the specification and the bar. Scoping an AI feature without it means guessing at both, and guesses are what pilots die on.

Before you scope the AI, do the work by hand. The rules you discover and the error rate you measure are the ones the model actually has to beat.