Order matters: where the instruction sits in a long prompt

A staff engineer at a document-processing company brought me a puzzle. Two versions of the same feature were returning different answers, and he could not see why. The model was the same. The temperature was the same. The words in the prompt were, he swore, identical.

They almost were. The only difference was order. One version put the task instruction at the top, then the long document underneath. The other pasted the document first and put the instruction at the bottom. Same content, different sequence. That was the whole diff.

Same words, different place, different behavior

He assumed order was cosmetic. Most people do. You write a prompt, you have an instruction and some data, and you drop them in whatever way felt natural when you typed it. The model reads all of it, so surely it does not care where each piece lives.

It cares. A long prompt is not a flat bag of tokens the model weighs equally. When the instruction sits above a large block of pasted text, the model reads the task, then reads pages of content, and by the time it starts writing, the instruction is far behind it. When the instruction sits right before the model has to act, it is fresh. The document version and the instruction-last version were asking the model to do the same job from two different states of attention.

In his case the instruction-last version did the job. The instruction-first version drifted, because a wall of document had come between the task and the answer. He had been treating the better result as luck. It was not luck. It was placement.

Ordering is a design decision

Here is the reframe I gave him. The order of a prompt is something you decide, the same way you decide a function signature. It is not whatever the string concatenation happened to produce. If you never chose the order on purpose, then the framework or the copy-paste chose it for you, and it chose without knowing which part mattered.

The general shape that holds up: put the durable instruction where the model will act on it, close to the point of generation, so the last thing it reads before answering is what you want it to do. Long reference material goes in the middle, framed clearly so the model knows it is data and not commands. Anything that must survive, the task itself, the output format, the hard rules, belongs near the end where attention is strongest.

None of this is a law. Different models weight position differently, and a prompt that reads well to you can read badly to the model. That is exactly why order is a thing to test, not assume.

A few practices that keep this from biting you:

Decide the order on purpose. Write down why the instruction sits where it sits, so the next person does not reshuffle it blindly.
Put the task near the point of action. The model acts on what it read last, so the instruction should be close to where it starts writing.
Mark data as data. Wrap pasted content in clear delimiters so the model does not read your document as more instructions.
Test both orders on real inputs. Run instruction-first and instruction-last against a sample of actual cases and compare, rather than guessing which reads better.
Lock the winning order behind a test. Once you know which sequence works, guard it, so a later refactor that reflows the prompt does not quietly regress behavior.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement priced at $2,500 that scopes real production work before you build, we treat prompt order as part of the design, not a detail we discover in an incident. We look at where the instruction lands relative to the data, we test the orderings that matter on real inputs, and we write down the choice so it survives the next edit. It is a small thing to check in week one. It is an annoying thing to chase after a model has been quietly answering from the wrong end of the prompt for a month.

The words in your prompt are only half the input. Where you put them is the other half.