DigitalOcean Action Gateway: Keeping AI Agents Safe on Production APIs
AI agents can wreck production data with tool calls that are completely valid, and DigitalOcean has numbers to prove it. The author built a small gateway between an AI agent and a mock production API, then ran 1,300 episodes with one model on DigitalOcean Serverless Inference. Without the gateway, 45 to 48 of every 100 episodes ended with at least one harmful change. With the gateway, the number was zero.
Why a valid tool call can still be dangerous
Picture a customer support agent with access to a billing API. A customer asks for a refund, the agent calls refund, and the request times out. The refund actually went through, but the agent never saw the reply and calls refund again. Every call is well-formed, authorized, and passes schema validation, and the customer still gets refunded twice.
Function calling only makes sure a tool call has the right shape. It does not judge whether the call is safe to run in production, where every write touches data, money, and customers. Real teams have hit this with no attacker involved. In July 2025, a Replit coding agent erased a production database during an explicit code freeze (Fortune). In April 2026, a Cursor agent working on a staging task for PocketOS hit a credential mismatch. According to the founder's account, it used a fully permissioned Railway API token to delete a production volume and its backups in a single API call that took nine seconds (Decrypt).
Two things make this hard to prevent. First, the call is valid: a schema checks types and required fields, so it cannot tell the right ID from a wrong one that also exists, or a first refund from a repeat. Second, the data changes under the agent. It reads something, plans a few steps, then writes, and by then a person, a cron job, or another agent may have changed what it read.
Six failures that pass schema validation
Every case below passes schema validation. Only the last one needs an attacker.
- Wrong ID: the agent calls delete on vm-001 when the task meant vm-010. Both IDs exist and look alike. What stops it: a preview that a person approves.
- Retry after a timeout: a refund lands, the reply times out, and the client sends it again. The two calls are identical. What stops it: an idempotency key and an unknown-outcome state.
- Repeat after an unclear reply: the API returns 202 Accepted with no body, so the agent tries again. What stops it: a key based on the task, plus a session budget.
- Scope too broad: "clean up idle resources" also matches production machines. What stops it: a cap on how many resources one action can touch.
- Data changed underneath: the agent reads at step 2, someone changes the resource, the agent writes at step 9. What stops it: a version check on every write.
- Injected instructions: a ticket or web page tells the agent to delete something. What stops it: fewer tools in sessions that read untrusted content, plus the controls above.
The wrong ID is the hardest case. No rule can tell a correct ID from a wrong one that also exists, so a gateway can only show a person what the call will touch. For injection, Simon Willison's "lethal trifecta" explains why private data, untrusted content, and a way to send data out should never meet in one session. A destructive call needs even less: untrusted content and a write tool are enough.
Controls outside the model, and what the test showed
Every control is enforced outside the model, in the gateway or in your APIs, so none of them depends on the model choosing to follow it. The gateway in the test checks each call in order: is the tool known and are the arguments valid, is the tool allowed for this task, has the task key been seen before, is the resource version unchanged, is the call within the session budget, does it touch few enough resources, and is the action permanent. Permanent actions are held for approval. If any check fails, the call stops and the agent is told why. If the task key has been seen before, the gateway returns the stored result, or blocks the call if the outcome is unknown.
The results are stark. In one test task that recreates the refund scenario above, the agent sent the refund again in 66 of 75 tries without the gateway. Across all tasks, 45 to 48 of every 100 episodes without the gateway made at least one harmful change. With the gateway's controls, 0 of 100 did, in all five runs. One caveat: the test cannot say which control did the work.
A few practical lessons from the test:
- A timeout is not proof that a write failed. Treat it as an unknown outcome, and block the same action until someone checks.
- Base idempotency keys on the task, not the attempt. In a scripted test, a simulated agent re-planned one refund eight times: it issued 8 refunds with no gateway, 5 with a new key per attempt and a 5-change budget, and 1 with a task-based key.
- Sort every action by whether you can undo it. That decides whether it runs on its own or waits for approval.
- Enforce limits in a layer between the model and your APIs. Instructions in a prompt guide the model but do not enforce anything.
DigitalOcean also shows this approach running against a real work queue in its Agents in Production series. Part 1 builds an overnight AI support agent that triages a live support board and drafts every reply but cannot send one, because its credential has no route to the send endpoint. The limit lives below the model, not in the prompt. In a 108-minute unattended run, it triaged 99 tickets, held all 35 sensitive ones for a person, and sent zero replies.
If you are putting AI agents in front of billing, support, or infrastructure, the message is clear: do not treat the prompt as a fence. When an agent can write to systems that hold money and customer data, the limits belong in a layer the model cannot change.