Agents turn tool outputs into data, and nobody is checking it

Everyone checks the data going into an agent and the answer coming out. The middle is where the risk actually lives. An agent calls a tool, takes the result, and feeds it into the next call. Each tool output is data for the next step, and almost nobody is validating it in flight.

The loop is a data pipeline you did not design

Think about what an agent does across a single task. It calls a search tool and gets rows back. It passes those rows to a calculation. It takes that number and calls an API. It reads the API response and drafts an action. Four steps, and at every hop the output of one tool became the input of the next.

That is a data pipeline. Nobody drew it, nobody put quality checks on it, and it assembles itself at runtime, differently for every request. In a normal ETL pipeline you would validate at each stage. Inside an agent loop, the model just trusts whatever the last tool handed it and keeps going. If step two returned a wrong number, step three does not question it. It builds on it. By step four the error is baked into an action, and the original bad output is three hops back where nobody is looking.

The failure is silent because each step looks reasonable

This is what makes it dangerous. Every individual step looks fine. The search returned rows, so the search worked. The calculation produced a number, so it ran. Nothing errored. The problem is that the rows were stale, or the number used the wrong column, and the model has no instinct that anything is off. It reads a confident-looking tool result and treats it as ground truth, because to the model, a tool output is ground truth. Fluency at each step hides a fault that only shows up in the final action, if it shows up at all.

Keep a human where the output becomes an action

An engineering leader who runs ad-serving infrastructure at a large platform, the kind answering tens of millions of requests a second, had a rule that maps straight onto this. An agent runs on the credentials of whoever started it, so whatever it does is done with that person’s access and lands on that person’s name. His hard line: an agent never commits its own code. A human stays in the path where the output turns into a real, irreversible action.

That is the right place to put the check. You cannot inspect every intermediate tool result in a long chain, but you can refuse to let the chain reach into production without a person signing the final step. The credentials belong to a human, so the accountability does. Keep the human where it matters, at the point the loop stops being internal and starts touching the world.

A named owner changes what “checking it” means

An engineering lead at a messaging startup put one senior engineer over a pool of agents. His surprise was that this gave the output a single named owner for the first time, instead of the vague sense that some agent somewhere produced the result. That matters here directly. If no person owns the agent, no person owns its tool outputs either, and “nobody is checking it” is literally true. Put a name on the pool and someone is now responsible for asking whether the data flowing between steps is any good.

A few concrete moves make the loop less blind:

Validate tool outputs the way you would validate an external data feed, with type and range checks, before the next step consumes them.
Log every intermediate result, so when the final action is wrong you can walk back and find the step that poisoned it.
Keep a human in the path where the loop produces an irreversible action, on the credentials of whoever started it.
Give every agent or pool a named owner, so the tool outputs inside it belong to someone.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two week, $2,500 engagement, we treat the agent loop as a data pipeline, because that is what it is. We map the tool calls, mark where one tool’s output becomes another’s input, and check whether anything validates those hops or whether the model just trusts them. We look for the point where the loop reaches production and confirm a human is standing there. The inputs and the final answer get watched. We watch the middle.

An agent that trusts its own tool outputs is a pipeline with no quality checks and no owner.