Data retention windows are quietly shrinking your training set

Two teams are pulling on the same data from opposite ends. Legal wants old records gone on a schedule. The model wants as much history as it can get. Both are doing their jobs. They almost never talk until the training set turns out to be thinner than anyone expected.

Deletion is a policy that runs whether you are watching or not

A retention policy is a promise. We keep this class of data for ninety days, or thirteen months, and then it is deleted. That promise is often a legal requirement, sometimes a contractual one, and it runs automatically. The rows leave on time whether or not a data scientist ever planned to use them.

The problem shows up when the same records are the raw material for a model. A demand forecast wants several years of seasonal history to learn the shape of a year. A churn model wants the full arc of accounts that eventually left. A fraud model wants the rare bad cases, which are exactly the old ones, spread thin across a long window. If retention quietly caps the history at ninety days, the model is learning from a keyhole view of a world that has been running for years.

Governance and data science optimize for opposite things

This is a structural tension, not a mistake by either side. Governance reduces risk by holding less data for less time. Every record you keep is a record you might have to defend, delete on request, or explain in an audit. Less history is safer.

Data science reduces error by holding more data across more time. Every record is signal, and the rare, old, unusual ones carry the most. More history is more accurate.

Left alone, these two forces resolve in whichever direction has the louder advocate that quarter. The retention policy ships, the deletion job runs, and eighteen months later a team tries to train a model and discovers the history it needed left the building on schedule, correctly, according to a policy nobody flagged as a data science decision.

The records that predate the schema

An engineering lead who has moved close to three billion records between business systems talks about data that predates the current schema, the old rows entered under rules that no longer apply. When he migrates, that history is a burden. He phases the work, runs the core entities first, lets a human reconcile, then reruns the tool for incremental records. But that same awkward old history is what a model would kill for, because it is where the rare cases and the long patterns live.

The lesson cuts both ways. The old data is expensive to keep and hard to handle. It is also the part with the most to teach. A retention policy set purely for compliance treats that history as pure liability and deletes it. A training plan set purely for accuracy treats it as pure asset and assumes it will be there. Both are half right, and the half they each miss is the half the other team owns.

Reconcile it before the policy hardens, not after

The reconciliation is not complicated once the two teams are in the same room. A few concrete moves:

List every model you might build in the next year, and what history each one actually needs, before you finalize a retention schedule.
Separate what must be deleted from what can be kept in an anonymized or aggregated form that survives the deletion rule while dropping the personal detail.
Decide retention per data class with both risk and future training value on the table, not risk alone.
Write down which datasets a deletion job will touch, so a data scientist is never surprised by a gap that a policy created on purpose.

The point is to make deletion a decision two teams make together, rather than one team’s policy that silently shrinks the other team’s raw material.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two week, $2,500 engagement, we look at retention as a data readiness question, because it decides how much history a use case can actually draw on. We map what the feature needs to learn from against what your deletion policies remove and when. Often there is a clean answer, an aggregated or masked form of the old data that satisfies governance and still feeds the model. We would rather find that in week one than have you discover the training set is half the size you assumed.

The deletion job does not know it is starving your model. Someone has to.