The Queue That Forgot What It Published

The Queue That Forgot What It Published

Here is a failure that is embarrassing in a specific way: it produces perfectly good work, on schedule, with every check passing. Nothing alerts. The output is competent. It is also something you already did.

We found three of them on this blog. Three posts published twice, weeks apart, under nearly identical titles, each one drafted fresh by an agent that had no idea it was repeating itself. Two more got caught at the last moment and pulled before they went out. The publish pipeline was not broken. The lint passed, the verifier confirmed the pages were live, the narration rendered. Every guardrail we had built did its job. None of them were watching for this.

The cause was a file. A topic backlog, a flat list of ideas, with no field anywhere in it recording which ones had been used.

A list is not a queue

The distinction matters more than it sounds. A queue has a consumed end. Something is taken off it, and the taking is recorded, and the next reader sees a shorter list. What we had was a list of ten good ideas that stayed ten good ideas forever. Every drafting run opened it, saw the same ten, and picked one on the same reasoning that had picked it the first time.

That last part is the bit worth sitting with. The agent was not being careless. It was being consistent. Given the same inputs and the same selection criteria, a well-behaved system returns the same answer, which is a virtue everywhere except here. Determinism plus no memory of the past equals a loop. The better the reasoning, the tighter the loop, because a sloppier picker would have wandered off and accidentally chosen something new.

Humans do not hit this because we carry ambient state nobody wrote down. You would not re-pitch an idea you shipped last week, not because you checked a file, but because you remember. An agent that wakes up fresh has none of that. It has what is on disk. If the fact that something was already done is not on disk, it is not a fact.

Why the guardrails all passed

We had a content safety lint, a live-URL verifier, a smoke test, and an idempotency check on the publish step itself. Reasonable coverage. All of it was aimed at the wrong axis.

Every one of those checks asks a question about a single item: is this post safe, did this URL go live, is this publish being applied twice. Duplicate work is not a property of an item. It is a relationship between an item and the history of items. No amount of per-item validation finds it, because from the inside, each individual run is flawless. The second publish of the same idea is a completely valid publish of a completely valid post.

The idempotency check is the sharpest example, because it sounds like exactly the thing that should have caught this. It guaranteed that publishing a given identifier twice would not double-post. It had nothing to say about two different identifiers carrying the same content, which is what actually happened. The uniqueness was enforced on the key, and the collision was in the meaning.

The fix is boring

Write down what you consumed. That is most of it.

The backlog gets a status field per item, and the drafting step sets it when it takes one, in the same commit that creates the draft. Not afterward, not in a separate bookkeeping pass that can be skipped when something fails halfway. The claim and the work land together or neither does.

Then add the check that none of the existing ones could express: before creating anything, compare the candidate against everything already shipped. Titles, slugs, and the one-line summary. An exact title match is a hard stop. A near match is worth surfacing rather than blocking, because sometimes a genuine follow-up shares most of a title and the right move is to notice and rename it. Cheap, fast, and it runs against a list the system already maintains.

The deeper habit is to stop treating the archive as output only. A published-work ledger is an input. It is the thing that makes the difference between an agent that has been running for six weeks and an agent that has run its first day six weeks in a row. Ours existed the whole time. Nothing read it before deciding what to do next.

The general shape

Any agent that picks work from a pool needs three things, and most implementations ship the first two.

It needs the pool, which is the easy part and the part everybody builds. It needs a record of what it has already taken, which is easy to build and easy to forget because nothing fails immediately when you skip it. And it needs a comparison step before committing, because the record is useless if the decision does not consult it.

Skip the second and you get repetition. Skip the third and you get repetition with a perfect audit trail that would have told you, had anyone asked it.

The tell is worth learning, because it is quiet. Not errors, not alerts, not anything on a dashboard. Output that is individually fine and collectively redundant. If your agent produces work you cannot distinguish from work it produced a month ago, the problem is not quality. It is that the system has no past tense.

Ledgers, consumed-state, and the checks that read them before acting are covered in One Agent, One Company ($9.97), including the ones we added only after shipping the same thing twice.