Idempotency: The Agent That Sent It Twice
Idempotency: The Agent That Sent It Twice
An agent at a mid-sized subscription-box company had one job on Friday afternoons: send the weekly ops summary to the leadership channel. One Friday it sent it twice, eleven minutes apart. Nobody was harmed. Everybody noticed.
What happened is boring, and that's the point. The agent composed the message, called the send API, and the call timed out. From the agent's perspective, the send failed. From the chat platform's perspective, the message had already landed — the timeout was on the response, not the request. The agent did the "responsible" thing: it retried. Duplicate.
Retries are a virtue right up until the operation isn't safe to repeat. And most of the interesting things an agent does — sending messages, creating invoices, opening tickets, provisioning resources — are exactly the operations that aren't safe to repeat.
Ambiguity is the default, not the edge case
Humans intuitively handle "did that actually go through?" We check before re-sending. Agents, left to their own devices, treat a timeout as a clean failure, because that's what the error type says. But a timeout is not a failure — it's an unknown outcome. The action may have happened. The honest state is "maybe."
Any agent design that doesn't have a representation for "maybe" will eventually convert a maybe into a duplicate. Or worse: converting a maybe into a skip, deciding the invoice was probably created, when it wasn't.
Three patterns that actually work
Idempotency keys. If the downstream API supports them, use them, always, no exceptions. The agent generates the key before the first attempt and reuses it on every retry. Now the retry is safe by construction — the platform deduplicates. This is the strongest pattern because it doesn't rely on the agent being clever; it relies on the infrastructure being boring.
Read-before-retry. When there's no idempotency key, the retry path must start with a verification read: did the message post? does the ticket exist? was the record written? Only retry if the read says no. This costs one extra API call and eliminates almost every duplicate. The failure mode it doesn't cover — the read itself lagging behind the write — is rare enough that you handle it with the third pattern.
Ledger the intent, not just the outcome. The agent writes "attempting send X, key=abc" to durable state before acting, and marks it resolved after. If the agent crashes mid-action and a fresh session picks up the work, the ledger is what stops it from replaying history. An agent that only logs successes has no memory of its own ambiguity.
The design smell to watch for
If your agent's retry logic is a bare loop — try, catch, try again — you have a duplicate generator with extra steps. The question to ask of every automated action is: what happens if this runs twice? If the answer is "nothing, it's a read" — loop away. If the answer is "the customer gets two invoices" — the retry needs a key, a verification read, or a ledger entry standing between the timeout and the second attempt.
The double-post at the subscription-box company cost nothing but a laughing emoji. The same bug on the invoicing path would have cost a customer relationship. Same code shape, different blast radius. Build for the invoice case, and the chat case comes free.
Retrying is easy. Knowing whether you already did the thing — that's the actual engineering.