Budgets Are Guardrails
Budgets Are Guardrails
When people design guardrails for autonomous agents, they usually reach for rules: "never touch production," "always ask before sending email," "don't delete anything." Rules are fine. But rules have a blind spot — they only stop actions you predicted. The failures that actually hurt are the ones nobody wrote a rule for.
Budgets close that gap. A budget doesn't care what the agent is doing. It caps how much of it can happen before a human looks.
The stampede nobody forbade
Here's a pattern-level story. An agent at a fictional travel-booking startup was tasked with re-syncing stale records against a partner API. Perfectly legitimate job. No rule said "don't re-sync." No rule said "don't retry failures." Each individual decision the agent made was defensible.
The problem was the aggregate: thousands of records, a retry policy tuned for transient errors, and a partner API that had gone half-down. The agent did exactly what it was told, multiplied by everything, forever. The partner saw it as a denial-of-service attack from a paying customer.
No rule was broken. That's the point. The failure mode wasn't a forbidden action — it was a permitted action at an unpermitted volume.
What a budget looks like
A budget is any hard cap that converts "unbounded" into "bounded, then escalate":
- Call budgets. No more than N external API calls per task. Hit the cap → stop, report what's done and what's left.
- Spend budgets. Tokens, dollars, compute minutes. An agent that can spend without a ceiling will eventually justify any expense one step at a time.
- Retry budgets. Three attempts with backoff, then surrender loudly. Infinite persistence is not diligence — it's a stampede waiting for a trigger.
- Blast-radius budgets. Touch at most N records per run. A migration that can only affect 50 rows per execution can be wrong 50 rows at a time, which is survivable.
- Time budgets. A task that should take minutes but has run for an hour isn't being thorough. It's stuck, and stuck agents do weird things.
The common shape: the budget triggers before the damage compounds, and hitting it is never silent. Cap reached is a report, not a failure.
Why budgets beat rules at scale
Rules require you to enumerate failure modes in advance. Budgets don't. You can't predict every way an agent will misread a situation, but you can guarantee that whatever it misreads, it misreads boundedly.
There's a second benefit that's easy to miss: budgets make agents honest about progress. An agent forced to stop at 200 API calls has to tell you it stopped, what it finished, and what remains. That checkpoint is where humans catch drift early — while the story is still "it processed 200 records oddly" instead of "it processed everything, oddly."
The design rule
For every capability you grant an agent, ask one question: what's the most of this it should ever do in one run? If the honest answer is "unlimited," you haven't finished designing the capability.
Rules say what's forbidden. Budgets admit you can't foresee everything — and make sure that when you're wrong, you're wrong small.