Map where the answer has to land before you scope the model

A head of data at a staffing agency told me about a candidate-screening tool that worked in every demo and failed in production for a reason that had nothing to do with the model. The tool read incoming resumes, scored them against a role, and wrote a short recommendation. In the demo, the recommendation appeared on screen. Everyone nodded. It was good.

Then they tried to ship it, and the question came up that had never come up in the demo. Where does the recommendation go. The recruiters lived in the applicant tracking system all day. They did not want to open a second tool to read AI notes. They wanted the score and the recommendation to appear on the candidate’s record, in the CRM, next to everything else they already looked at.

That destination turned out to be the hard part. The CRM had a field for a score, but it was a dropdown with fixed values, and the model produced a number on a different scale. The recommendation had to fit a notes field with a character limit nobody had measured. Writing into the record required an integration path the team had not scoped, because during discovery they had been staring at the model, not the place the answer had to land.

The demo lies about the last mile

A demo shows the answer on a screen built for the demo. Production has to put the answer where the work actually happens. Those are different problems, and the gap between them is where features go to die.

The model producing a good answer is maybe half of a working feature. The other half is the answer arriving in the system the user already trusts, in a shape that system accepts, at the moment the user needs it. If the recommendation lands in an inbox nobody checks, or a dashboard nobody opens, or a field that rejects its format, the quality of the answer is irrelevant. It stranded.

A head of support at a software company described the same failure with a ticket-triage tool. The model classified tickets beautifully. The classification had to write back into the help desk to actually route the ticket, and the help desk’s write path was rate-limited in a way the team discovered the week before launch. The intelligence was done. The plumbing was not, and the plumbing was the feature.

Map the destination during discovery

The fix is boring and it works. Before you scope the model, name exactly where the answer has to go and study that place as carefully as you study the model.

Which system holds the record the user acts on. The CRM, the queue, the help desk, the inbox.
What shape does that system accept. Fixed field values, character limits, required fields, allowed formats.
How does the answer get written in. An API, a webhook, a manual paste, a nightly sync.
Who or what is watching that destination. If the answer arrives and nobody looks, it did not arrive.

None of these are model questions. All of them decide whether the model’s output becomes work someone acts on or a result someone admires and ignores.

The staffing agency eventually shipped, but only after they redesigned the score to match the CRM’s dropdown and built the write path they had skipped. Had they mapped the destination during discovery, that redesign would have been a decision on day two instead of a rebuild in month two.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week scoping engagement at $2,500, we trace the answer all the way to where a human acts on it. Not the demo screen. The real record in the real system, with its real constraints. We check the shape it accepts and the path that writes into it, because that path is usually where the surprise cost hides. Scoping the model without scoping the destination is scoping half a feature, and it is the visible half, which makes it the more tempting one to mistake for the whole.

An answer with nowhere to land is not a feature. It is a demo that ran out of road.