The retry tax: what error handling costs at volume

Retries are good engineering. A model call fails, times out, or returns something malformed, and a sensible workflow tries again rather than falling over. The logic is sound. The cost is invisible. Every retry is another paid call, and the calls that need retrying are often the expensive ones to begin with. At low volume this is noise. At production volume the retry tax can be a meaningful share of the bill, and almost nobody has it on a line of its own.

The tax that lives in the code path

A platform engineering lead at a SaaS company found his retry tax by accident. He was reconciling his inference bill against his logged successful responses and the numbers did not match. The bill was clearly higher than the successes could explain. The gap was retries. His workflow was quietly trying failed calls up to three times, which was the right behavior for reliability, and each of those attempts was a paid call that produced nothing a user ever saw. On the hardest inputs, the ones most likely to fail, he was sometimes paying three times over for a single answer. His success dashboard was honest. It just did not count what he paid to fail.

The tax compounds where retries and complexity meet. An agentic workflow can already use ten to thirty times the tokens of a simple call for one user intent, and a retry does not restart from the last step. It often reruns a chunk of the chain. So a retry inside an agent is not one extra call. It can be several, at the expensive end of your token spend, since output tokens run around five times the cost of input tokens on a Sonnet-class model. The failures you handle gracefully are, in cost terms, the least graceful part of the whole system.

A VP of engineering at a mid-market logistics company had learned to treat retries as a budget item, not a code detail. His rule was that any retry policy needed a cap and a counter. The cap kept a bad input from retrying forever and running up a bill on its own. The counter made the tax visible, so he could see when a spike in failures was quietly becoming a spike in spend. Without the counter, he said, the retry tax is the one cost you pay every month and never see, because it hides inside the word reliability.

Why it stays hidden

The retry tax hides because retries are supposed to be invisible. The whole point of the pattern is that the user never notices the failure, so the system is designed to swallow it. That design instinct, applied to the bill, means the cost gets swallowed too. You see the successful answers on your dashboard and you pay for the failed attempts on your invoice, and the two numbers live in different places, so almost nobody lines them up.

Making the tax visible

To keep retries from quietly inflating the bill, instrument these:

Retry rate. The share of calls that get retried at all, watched as a trend, since a rising rate is a rising cost.
Attempts per success. The average number of paid calls behind one delivered answer, which is the real cost multiplier.
The retry cap. A hard limit on attempts, so a single bad input cannot run up an open-ended bill.
Failure-cost tracking. A line that counts spend on calls that produced nothing, separate from spend on successful answers.

How we approach it at Density Labs

When we map a feature’s cost, we count the failed attempts, not just the successful answers. Retry behavior is part of the unit cost, so we measure attempts per success and put a cap and a counter on the policy. That turns the retry tax from an invisible drain into a number you can manage, which is usually the difference between a reliability feature that pays for itself and one that quietly does not.

Reliability is not free. Every retry is a paid call, and the ones you retry are the expensive ones. Measure the tax, cap it, and it stops being the cost that shows up on the invoice and nowhere else.