Prompt length is a cost, a latency, and a bug surface

A staff engineer at a media company pulled up a prompt that had gotten out of hand. It was thousands of tokens long. He scrolled through it and could not tell me, off the top of his head, what half of it was for. Every line had been added by someone, for a reason, at some point. Nobody had ever taken a line out.

The prompt worked. It also cost the team on every single call, in three ways at once.

The prompt grew by patching

The history was familiar. A user hit an edge case, so someone appended a paragraph to handle it. A weird input slipped through, so someone appended a rule. A new requirement landed, so someone appended a section. Appending is the fastest fix available. You do not have to understand the rest of the prompt to add to the end of it, so that is what everyone did, over and over, for a year.

The result was a prompt nobody had designed. It had been accreted. It handled a hundred cases, some of which no longer occurred, with instructions that in a few places contradicted each other. And because every one of those tokens went to the model on every request, the team paid for the whole pile every time, whether the request needed it or not.

Every token is paid on every call

This is the part that is easy to forget. A prompt gets sent to the model on each request, in full, every single time. So the length of your prompt is a per-call cost, three times over, and never a one-time authoring cost.

It costs money. Every token in the prompt is billed on every call, so a bloated prompt multiplied by production traffic is a real line on the bill, spent on instructions that many requests do not use.

It costs latency. A longer prompt takes longer to process. The user waits a little more on every request, and that wait scales with the length you never trimmed.

It costs correctness. This is the one teams underrate. A longer prompt has more places for instructions to conflict, more rules competing for the model’s attention, more room for a new line to quietly undermine an old one. Length is a bug surface. The bigger the prompt, the harder it is to reason about what it will actually do, and the easier it is for two of its rules to disagree.

None of these show up as a failure the day you append the line. They accumulate, and by the time anyone notices, the prompt is too big to hold in one head.

Keep it tight on purpose

A prompt stays lean only if someone keeps it lean. That is ordinary engineering discipline applied to a file most teams never refactor.

Refactor instead of appending. When you add a behavior, fold it into the structure rather than tacking a paragraph onto the end.
Remove dead instructions. Rules added for problems you no longer have are pure cost, in tokens, latency, and confusion.
Consolidate overlap. Several instructions saying nearly the same thing become one clear one.
Factor shared context deliberately. Pull common instructions into one place rather than repeating them, and send only what a given call needs.
Budget the length. Treat a growing prompt as a signal to clean up, the way you would treat a function that keeps gaining parameters.

The goal is a prompt you can read in one sitting and reason about with confidence, because that is also the prompt that costs the least and fails the least.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement priced at $2,500, a prompt that has ballooned to thousands of tokens is one of the clearest things to fix. We find the instructions that never fire, the ones that conflict, and the sections that repeat, and we cut the prompt back to what the feature actually needs. Shorter is usually cheaper, faster, and more reliable at the same time, which is a rare thing to get for free.

Every token in your prompt is paid on every call. Spend them like they cost something.

A prompt you cannot read in one sitting is a prompt you cannot reason about.