Relevant beats recent when you assemble context

A data engineer at a fleet-management company had a feature that answered questions about vehicles and routes. To build the context for each question, it grabbed the most recent records for that vehicle and handed them to the model. Simple, fast, and wrong often enough to be a problem.

The complaints followed a shape. A user would ask something specific, and the model would answer confidently using data that had nothing to do with the question. The answer was fluent. The facts were real. They were just the wrong facts, because the context was assembled by time, and the question was not about time.

Recency is a convenient proxy, not a good one

Grabbing the newest records is easy to build. You sort by timestamp, take the top slice, done. And it feels reasonable, because recent data is often what people care about. So teams reach for recency as a stand-in for relevance without noticing they made a substitution.

The substitution holds right up until someone asks a question that is not about the latest events. Ask why a particular route ran long last month, and the ten newest records for that vehicle tell the model nothing useful. They are fresh and irrelevant. The model, handed fresh irrelevant context, does the only thing it can. It answers from what it was given, and produces a clean paragraph about the wrong thing.

The failure is quiet because both halves look fine. The data is genuine. The prose is coherent. Nobody sees that the retrieval step selected on the wrong axis. It reads like the model misunderstood the question, when really it understood a different context than the question needed.

Assemble by relevance to the question

The correct move is to select context by how well it answers the question in front of you, not by when it happened. Recency can be one signal among several, and for some questions it genuinely matters. But it is a weak default. The primary axis has to be relevance: what content actually bears on what the user asked.

That means the question has to shape retrieval. A question about a specific incident should pull the records tied to that incident, whenever they occurred. A question about current status should pull recent state. The point is that the selection follows the intent of the question, rather than always reaching for the newest rows regardless of what was asked.

How to put relevance first:

Select on the question, not the clock. Let what the user asked drive which records you pull, so context matches intent instead of timestamp.
Demote recency to one signal. Keep it in the mix where it helps, but stop treating newest as a synonym for most relevant.
Check the retrieved set against wrong answers. When an answer is off, look at what context was assembled, because the cause is usually visible there in seconds.
Test questions that are not about the present. Include cases where the right answer lives in older records, since those are exactly the ones a recency default gets wrong.
Match the selection to the question type. Route status questions to recent state and history questions to the relevant past, rather than using one rule for all.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement priced at $2,500 that scopes real production work before you build, we look closely at how context gets assembled, because this is one of the most common ways a feature produces confident wrong answers. We check whether retrieval selects on relevance or on some convenient proxy like recency, and we test the questions where those two come apart. Finding that the context was assembled on the wrong axis takes a sample of real questions and an afternoon. Finding it after users have stopped trusting the feature costs a lot more.

Recent is easy to sort by. Relevant is what the question actually needs. Do not confuse the two.