The instruction the model obeyed until you added one more

A support automation lead at a telecom company came to me with a rule that had worked for months and then stopped. The model had always ended its replies with a specific closing line. Reliable, every time. Then, without any change to that part of the prompt, it started leaving the line off, seemingly at random.

Nothing about the closing instruction had changed. Everything around it had.

Every addition was reasonable

Their prompt had grown the way most prompts grow. Someone noticed the model was too formal, so they added a line about tone. Someone noticed it skipped a disclaimer, so they added a rule requiring it. A new product launched, so they added handling for it. Each addition was sensible on its own, added by someone solving a real problem, and each one made the prompt a little longer and a little more crowded.

The closing line had lived near the top of the prompt, back when the prompt was short and the instruction had room. As more rules piled in around it, it started competing for the model’s attention with a dozen other demands. Some replies still ended correctly. Some did not. The instruction had not been removed. It had been drowned out.

The team had assumed the prompt was additive. You want a new behavior, you add a line, and the old behaviors keep working. That is not how it goes. Instructions compete, and past a certain density the model starts quietly choosing which ones to honor.

Instruction budget is real

A prompt does not have infinite room for rules. As you add more, the model’s attention spreads thinner across them, and the ones that used to be reliable become intermittent. There is a budget, and most teams overspend it without noticing, because the cost does not show up as an error. It shows up as an old rule that used to hold and now only holds sometimes.

The trap is that each new instruction looks free. It is one more line, and the prompt still produces output. The bill comes due later, on a different instruction, one nobody touched, which starts failing because a newer rule crowded it out. That delay is what makes the cause so hard to find.

Managing the budget is ordinary work once you accept it exists.

Prioritize. Decide which instructions are non-negotiable, and make sure those are stated clearly and early.
Consolidate. Fold overlapping rules into one. Five vague lines about tone can usually become one clear one.
Remove what is stale. An instruction added for a problem you no longer have is just noise competing with the rules you still need.
Test the old rules after adding a new one. Confirm the behaviors you already relied on still hold, because a new line can silently break them.
Watch the length. When a prompt keeps growing, treat that as a signal to refactor rather than keep appending.

The point is to stop adding instructions as if they were free and start spending the budget on purpose.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement priced at $2,500, a long and heavily patched prompt is one of the first things we look at. We check which instructions actually still fire, which ones compete, and which ones were added for a problem that no longer exists. Then we run a real sample and confirm the rules the team depends on hold under the full prompt rather than in isolation.

A prompt is not a place where instructions accumulate for free. When you add one, check that the old ones survived it.

Every rule you add costs attention the earlier rules used to have.