Versioning prompts like code, because they are code

A platform team at a logistics company had an incident with their AI feature. For a stretch of one morning, the assistant started giving answers that were confidently wrong about delivery windows. By the time someone noticed and it settled down, the obvious question came up in the review: what changed right before this started? Nobody could answer it. Not because they were careless, but because there was no place that recorded the answer.

The prompts lived in three places. The main system instruction was a string in the application config. The few-shot examples were rows in a database table that a product manager could edit through an internal dashboard. And a chunk of dynamic instruction was assembled in code at request time from feature flags. Each of those could change independently, none of them kept history, and none of them were tied together. So when behavior shifted, there was no single log to read. The team spent the incident review guessing.

The thing controlling the model’s behavior had no version history. That is the whole problem in one sentence.

If it changes behavior, it needs the same history your code has

A prompt is not documentation and it is not content. It is the instruction set that determines what the model does. That makes it functionally the same as code, and it deserves the same treatment: it lives in version control, changes go through review, and every version is recoverable with a record of who changed it and why. When those three things are true, “what changed?” is a one-line answer instead of a morning of guessing.

The scattered setup fails all three. Config strings edited in place have no diff. Database rows edited through a dashboard have no review and often no history. Instruction assembled from flags at runtime cannot even be reconstructed after the fact, because the flag state that produced it is gone. Each piece was invisible on its own, and together they were unaccountable.

We consolidated. The system instruction, the few-shot examples, and the assembly logic all moved into the repository as versioned files. Changes went through pull requests, so a human reviewed them and the diff was preserved. The dashboard that let a product manager edit examples directly was replaced with a flow that opened a reviewed change instead of writing to production. And every deploy recorded which prompt version was live, so any future incident could be tied to an exact version.

What good teams do

Keep prompts, few-shot examples, and system instructions in the repository, not in ad hoc config, database rows, or runtime assembly with no record.
Put prompt changes through review, so a second person sees the diff and the change is preserved.
Record provenance: who changed it, when, and why, tied to the change itself.
Stamp every deploy with the prompt version that went live, so behavior can be traced to an exact version after the fact.
Make any prior version recoverable in one step, so rollback is mechanical, not archaeological.

The next time the logistics team saw a behavior shift, the review took minutes. They read the deploy log, saw which prompt version went live just before, opened the diff, and found the changed example that caused it. The fix was obvious once the cause was visible, and it was only visible because the history existed.

How we approach it at Density Labs

Prompt provenance is one of those things nobody misses until an incident makes it urgent, and by then the history you wish you had does not exist. Our AI Opportunity Assessment is a fixed two-week engagement, priced at $2,500, where we scope the real production work before you build. We find every place a prompt actually lives in your system, including the ones assembled at runtime that nobody thinks of as a prompt, and design a way to version all of it with review and provenance. You leave with a plan to make “what changed?” answerable.

Anything that controls how your model behaves is code, whether or not it looks like it. Give it a history, and your next incident review is a lookup instead of a guess.