The token bill as a production alert, not a month-end surprise

A staff engineer at a SaaS company told me about the week her team shipped a small improvement. They added more examples to a prompt so the model would follow the format better. It worked. Output quality went up. Everyone moved on.

Nobody was watching cost. The extra examples went out on every single request, and the feature ran on high volume. The spend climbed steadily and silently. The team found out at the end of the month, when finance forwarded the provider invoice with a question mark next to it.

The change was reasonable. The output was better. The problem was that a production variable moved a lot and no signal fired.

The lesson: instrument per-feature token cost and alert on anomalies

Teams already know how to watch latency and error rate. A spike pages someone. A regression shows up on a dashboard. Token cost deserves the same treatment, because it moves for the same reasons: a code change, a traffic change, or a bug. The difference is that cost usually fails silently. A retry loop or a bloated prompt does not throw an error. It just spends more, and the only alarm is a bill that arrives weeks later.

So make cost a metric, not an accounting line. For each AI feature, track token usage per request and roll it up per feature and per day. Providers return token counts in their responses, so the raw data is already in your hands. You just have to record it next to the call and graph it.

Then alert on movement. If a feature’s cost per request jumps, or its total daily spend crosses a threshold, someone should hear about it the same day, not at month end.

What good teams do

Record input and output token counts on every AI call, tagged by feature.
Build a per-feature daily cost view so a change is visible against its own baseline.
Alert on cost per request, which catches prompt bloat even when traffic is flat.
Alert on total daily spend, which catches runaway loops and traffic spikes.
Put a cost check in the review of any prompt change, since a small edit can multiply spend on every call.
Set a hard ceiling or rate limit as a backstop, so a bug cannot run unbounded overnight.

The retry loop deserves its own mention. A common pattern is to retry a failed AI call, which is sensible. Without a cap, a transient provider problem can turn one request into many, and multiply cost across every affected user at once. A cost alert catches that within hours instead of weeks.

How we approach it at Density Labs

Cost blindness is one of the quieter ways AI features get into trouble in production. The feature works. Users are happy. The economics quietly break, and the first real signal is a number nobody budgeted for. It fits the broader pattern where most enterprise AI efforts stall not on model quality but on the operational plumbing around the model, and cost instrumentation is part of that plumbing.

Our AI Opportunity Assessment is a fixed two-week engagement for 2,500 dollars that scopes the real production work before you build. For a feature that calls a paid model, part of that scope is confirming that token cost is measured per feature, that a prompt change cannot silently multiply spend, and that a loop cannot run unbounded. We show you where a cost alert belongs and what threshold makes sense for your volume.

Your model spend is a production metric. Watch it live, or read about it on the invoice.