The Rollback Budget: How Many Unreviewed Agent Actions Can You Reverse Before a Customer Sees One?

A pilot with a rollback budget of zero is a demo with production credentials.

That sounds harsh until you watch it happen. The eval dashboard is green, accuracy looks great, and the agent sends one wrong renewal quote to a real customer. The dashboard is still green. The customer's inbox is not.

Accuracy tells you how often the agent is right. It says nothing about what happens the one time it isn't. That second question has a number, and most teams have never counted it.

What is a rollback budget for AI agents?

The rollback budget is the number of agent actions that can run without human review, get caught, and be fully reversed before a customer ever sees the result.

It isn't a confidence score or an error rate. It's a property of the workflow around the model: what the agent is allowed to touch, how fast you notice a bad output, and whether you can actually undo it. A mediocre model inside a well-staged workflow can carry a healthy budget. A brilliant model with direct write access to customer email has a budget of zero, and no amount of prompt tuning changes that.

If you can't state the number, assume it's zero.

How to count your rollback budget: a builder's checklist

You can run this against your pilot in an afternoon with a whiteboard and whoever wrote the tool definitions.

1. List every action the agent can take

Not what it usually does. What it can do. Go through the tool and function definitions line by line:

• Write or update a record in the CRM or policy system
• Send an email or message
• Update a quote, price, or contract field
• Call an external API (payments, scheduling, a partner system)
• Create, close, or reassign a ticket

Teams are routinely surprised here. The tool that was "only for reading" turns out to have an update method somebody wired in during week two.

2. Mark each action reversible or irreversible

Can you restore the prior state exactly, with an audit trail? A record write with versioning is reversible. A sent email is not. An API call that triggers a charge or a shipment is not, at least not cleanly.

3. Mark each action staged or customer-facing

Staged means the output lands somewhere internal first: a queue, a draft, a review table. Customer-facing means the next step is a human outside your company seeing it.

4. Measure time to detection

How long between a bad output and someone (or something) noticing? Minutes, because an eval check runs on every output? Days, because you find out when the customer calls?

5. Score it

The rule is simple: irreversible plus customer-facing plus slow detection equals a budget of zero for that action. Any one of the three can be survivable. All three together means every unreviewed run is a bet with the customer relationship as the stake.

Some workflows will score well. An agent drafting internal summaries into a review queue can absorb plenty of mistakes. Some will score terribly, and that's the point. The checklist doesn't tell you AI is bad for that workflow. It tells you exactly which action needs a gate before go-live.

Why a zero rollback budget is a security problem

For CISOs: an unbounded rollback problem is almost always an excessive-agency problem, which the OWASP LLM Top 10 lists by name. The agent has more permissions, more autonomy, or more functionality than the task needs. A prompt injection or a plain hallucination then has a direct path to an irreversible, customer-visible action.

The fixes are unglamorous and they work:

• Scoped permissions. If the task is drafting quotes, the agent gets draft access, not send access. We cover this in depth in defining data boundaries before go-live and scoping RAG permissions across MCP agents.
• Staging queues. Convert customer-facing actions into staged ones. The agent writes; a release step publishes.
• Human approval gates on irreversible actions only. Gate everything and you've rebuilt the manual process with extra latency. Gate the sends, charges, and contract changes.
• Evaluation harnesses with quality gates. Run evals on every candidate release and block promotion when any action still scores a budget of zero.

On the governance side, this maps cleanly onto how NIST AI RMF and ISO 42001 already think. Counting actions and their impact is the "map" work. Detection time is "measure." Gates and scoped permissions are "manage." You don't need a compliance lecture to use it, but your auditor will be happy you can show the table. If your governance doc stops at the model and never reaches individual agent actions, that gap tends to show up on workflow two.

How we make the rollback budget a go-live criterion

In a Pilot-to-Production Build (6 to 10 weeks, milestone-based), the rollback budget is a go-live criterion, not a postmortem item. We count it during the first milestone, design the staging and gates around the actions that score zero, and don't promote anything to production until every customer-facing action has either a non-zero budget or a human in front of it.

This is also what makes speed safe. PolicyFlow, our AI-powered insurance data intake build, was shipped by one senior engineer plus an AI co-pilot in 3 months, work that would have taken a 4-person team 12 months. That kind of pace only holds up when reversibility is designed in from the start. Ship fast without it and you just reach the irreversible mistake sooner. You can see more of that shipped work on our ventures page.

The ordering matters. Teams that bolt on approval gates after an incident usually bolt on too many, and the workflow quietly slides back toward manual. Teams that count first gate precisely.

What belongs in next year's AI budget

Fall planning is when pilots get funded or quietly shelved. A pilot that can't state its rollback budget is a weak line item, because nobody can say what production will cost when it goes wrong. A pilot that can say "these four actions are reversible and staged, these two are gated, detection runs on every output" is a fundable plan.

If you count and come back with zero, that's not failure. It's the most useful number you'll produce this quarter, and it's exactly what the AI Opportunity & Readiness Sprint (2 weeks, fixed-fee) is built to work through before any money moves to production.

Start with the count. Grab the rollback-budget worksheet, run it against your pilot this week, and see which actions are carrying risk you haven't priced in. If the answer is zero, book a Readiness Sprint at https://horizontwolabs.com and we'll work out what it takes to make it non-zero.