The reconciliation step your migration skipped is the one AI needed

There is a step in a serious data migration that never makes it into the demo. A human sits down between phases and reconciles the data by hand. It is slow, it is unglamorous, and it is the step pilots quietly cut. It is also the one the AI feature depended on.

How a careful migration actually runs

An engineering lead who has moved close to three billion records across retail, restaurant, and accounting systems does not load data in one pass. He phases it. Core entities first, customers and products and the relationships between them. Then a human reconciles what landed, resolving the conflicts the machine could not decide on its own. Only after that reconciliation does he rerun the tool for the incremental records and go live.

That middle step is the whole point. The first load gets the bulk of the data across. The reconciliation is where a person handles the cases that have no automatic right answer, the ones where two systems disagree and something has to choose. Skip it and the data technically moved. It just moved every unresolved conflict along with it.

The conflict a machine cannot resolve alone

His clearest example is the duplicate customer. One source system lets you create the same customer twice. The next system does not allow duplicates at all. So the same person exists twice in the source and can exist only once in the destination, and something has to decide which of the two records survives. His platform defaults to last write wins, but the point he keeps returning to is that a human had to design that rule. Last write wins is not a law of nature. It is a decision someone made, on purpose, during reconciliation, because the alternative was letting the machine guess.

Now picture the pilot version. The curated slice was deduplicated by hand before the demo, so the duplicate never appeared. The model looked great. Then production starts flowing, the duplicate arrives on day one, no reconciliation rule was ever designed, and the machine picks a record at random or merges them wrong. In a restaurant integration, that resolves into an order that silently never reaches the kitchen. No error. Just a customer waiting for food that no system knows to make.

Why pilots skip the step that matters most

Reconciliation gets cut for an understandable reason. It is human, it is slow, and it makes the timeline honest in a way that is uncomfortable. Cutting it makes the pilot faster and the demo cleaner. The cost is deferred, not removed. Every conflict the reconciliation pass would have caught is still in the data. It is just waiting for a real customer to trigger it in production, where it costs a support ticket and a lost order instead of an afternoon of a person’s attention.

This is the same shape as the broader pilot trap. Pilot data is a clean curated subset. Production data is not. Reconciliation is specifically the work of turning the second into something as trustworthy as the first, and it is exactly the work a rushed pilot pretends it can skip.

Reconciliation is a design task, not a cleanup task

The reframe that helps: reconciliation is not janitorial work you do at the end. It is a design task you do in the middle. Someone has to decide, for each kind of conflict, what the rule is. Which record wins when two disagree. What happens when a required field is empty in one source and present in another. How to merge two half-complete versions of the same entity into one that is whole. Those rules are the reconciliation. They do not write themselves, and a model handed unreconciled data will invent its own rules invisibly, one wrong answer at a time.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two week, $2,500 engagement, we look for the conflicts a reconciliation pass would resolve before you build on top of the data. Where does the same entity live in two systems, which fields disagree across sources, and what rule decides the winner. If no one has designed those rules, the model will improvise them, and improvised rules are the ones that drop an order without a trace. We would rather surface the duplicate-customer problem in week one than watch it become a missing shipment in month three.

The reconciliation step is where the conflicts go to get decided. Skip it and they get decided in production, by no one, in front of a customer.