Scope to the input you have, not the one you wish you had

A data lead at an insurance carrier walked me through a project that looked healthy right up until it touched reality. The team wanted to automate part of claims intake. The design assumed each claim arrived as a clean structured record. Policy number here, loss date there, damage category in a tidy field. Given that input, the AI’s job was straightforward, and the demo they built on sample data was convincing.

Then they pointed it at the actual inbox. What arrived was not structured records. It was scanned PDFs of handwritten forms, photos of documents taken at an angle in bad light, and long free-text emails where the important detail was buried in the third paragraph. The clean fields the design assumed did not exist anywhere. Someone downstream was creating them by hand, which was the very work they were trying to automate.

The project did not fail because the model was weak. It failed because the whole thing was scoped against an input that did not exist.

Teams scope for the input they wish they had

This happens constantly, and it is easy to understand why. When you sketch an AI feature, you picture the ideal case. Structured, complete, consistent. It is the input that makes the design elegant, so that is the input the design assumes. The problem is that the ideal input is usually the output of a manual process someone is already doing. If clean structured data existed, a human made it clean, and that human is often the person the project is meant to relieve.

The real input is whatever shows up before anyone has cleaned it. Messy PDFs. Free text. Half-filled forms. Three formats because three teams did intake differently for a decade. That is the input your AI actually has to handle, and it changes the scope completely. Reading a structured claim field is trivial. Pulling a loss date out of a blurry photo of a handwritten form is a different and much harder task, and it may not be worth doing at the accuracy you would need.

A finance operations lead at a subscription company told me about the same wall from another angle. Her team scoped an AI tool to categorize expenses, assuming every transaction came with a clean merchant name and category. The real feed had merchant strings like garbled payment-processor codes, the kind of thing that means nothing to a human either. The tool could not categorize what a person could not read. They had scoped for the clean feed and been handed the raw one.

Look at the real input before you commit to anything

The fix is not clever. It is a discipline. Before you scope an AI feature, go get a real sample of the input as it actually arrives, unedited, and look at it. Not a curated demo set. The genuine mess, including the weird edge cases people forget to mention.

When you do, you learn things that change the plan:

Whether the signal you need is even present in the raw input, or whether a human adds it later.
How many formats you are really dealing with, and how ugly the worst ones are.
Whether the hard part is the AI decision or the extraction step before it.
What fraction of inputs are clean enough to handle now, so you can scope to those first.

That last point is the useful one. The insurance team did not have to solve every input. When they looked at the real mix, a meaningful share of claims did arrive in a clean digital form through the online portal. They scoped the first version to only those, handled the messy paper claims the old way for now, and shipped something real instead of stalling on the hardest ten percent.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement at $2,500, we insist on seeing the real input before we scope anything. We ask for a raw sample, the actual files and feeds as they land, and we read through the ugly ones on purpose. That is usually where a project’s real difficulty lives, and it is far cheaper to find it in week one than after you have built against an input that was never going to arrive.

Do not scope your AI feature for the clean input you imagine. Go find the messy input that actually shows up, and scope for that.