Idempotency for AI: what happens when the model runs twice

An operations lead at a marketplace company described a strange week where customers started reporting duplicate notifications, and sales saw duplicate records appear in their CRM. The AI feature that generated and sent these was, by every log they checked, working correctly. It was working correctly. It was also running twice for some events, and the feature had no idea that the second run was a repeat of the first.

Duplicates are the normal state of a real system

It is tempting to assume each event happens once. Real systems do not work that way. A network blip causes a retry. A queue delivers the same message more than once, which most queues can and do. A user double-clicks. An upstream service replays events after a restart. In a distributed system, the same event arriving more than once is not an edge case you might hit. It is the normal weather you are guaranteed to hit.

For a read-only feature this is harmless. Summarize the same record twice and you get the same summary. Nobody cares. The moment the AI action has a side effect, duplication turns into damage. If the action writes a record, you get two records. If it sends a notification, the customer gets two messages. If it creates a task, someone works it twice. The model did nothing wrong on either run. The problem is that the second run had no way to know it was a repeat, so it did the whole thing again.

This gets sharper with AI actions specifically, because the work is often expensive and outward-facing. A duplicated model call costs money twice. A duplicated write pollutes a system of record. A duplicated message reaches a real person and erodes trust. And because each individual run looks correct in the logs, the cause is easy to miss. Everything is behaving. There are just two of everything.

Make the action safe to repeat

The lesson is that any AI action with a side effect must be safe to run more than once, because sooner or later it will. That safety comes from idempotency: attach a stable key to the unit of work, and before the action takes effect, check whether that key has already been handled.

What that looks like in practice.

Derive a stable idempotency key from the event, so the same event always produces the same key.
Check that key before writing or notifying, and skip the action if it has already run.
Deduplicate at the write, so a repeat updates or no-ops instead of inserting a second record.
Guard outbound side effects, notifications and messages, with the same key, so a person is contacted once.
Record which keys have been processed, so a replay after a restart is recognized as a replay.

This is standard discipline for any system that writes or notifies, and an AI action is no different, except that its side effects tend to be costlier and more visible. Build the action assuming it will be called twice, and the day it is called twice becomes a non-event instead of a support queue.

How we approach it at Density Labs

Most AI features that create duplicates are not buggy in the usual sense. They are correct actions with no memory of having already run, fired again by a retry or a replayed event that every real system produces. The model behaved. The design assumed each event happened exactly once, and no event ever promised that.

Our AI Opportunity Assessment is a fixed two-week engagement, priced at $2,500, that scopes the real production work before you build. Part of that is deciding idempotency and deduplication for every AI action that writes or notifies, so a duplicate event costs you nothing instead of costing you a duplicate record and a customer’s trust.

Assume the model will run twice. Design so the second run does nothing.