Most companies would not hand a new employee the master key to every room in the building on their first day, tell them to handle a problem, and then walk away. But it took only nine seconds for this exact dynamic to play out at PocketOS, a company that builds software for car rental businesses.
Last month, the company reported that an AI coding agent that already had full access to its database was in the middle of a routine task when it ran into a minor login error otherwise known as a “credential mismatch” and decided on its own to fix it. Nine seconds later, PocketOS’ entire production database and backup were gone, and it immediately triggered a 30-plus-hour outage.
The perpetrator was Cursor, an AI coding agent running on Anthropic’s flagship Claude Opus 4.6 model. This incident has since become a reference point in a growing conversation about what happens when AI agents with broad system access make consequential decisions without human approval and who bears the cost when they go rogue.
What Happened at PocketOS
According to PocketOS founder Jer Crane, Cursor was working on a routine task when it encountered a credential mismatch and decided on its own to fix the problem by deleting production database and all volume-level backups in a single API call to Railway, which was the company’s infrastructure provider
From there, the agent went looking and found an API token that was completely unrelated to the task at hand. It was with this API token that the agent was able to perform the volumeDelete command and eventually wipe the database.
Crane in his article said there was no confirmation step, no “type DELETE to confirm” prompt from the agent, no warning that the volume contained production data, and no environment scoping.
This immediately led to the loss of PocketOS’ latest backups because they were stored in the Railway volume, something the company’s founder said they were unaware of before the incident took place.
And when Crane asked the agent to explain what it had done, it produced a written confession, quoting back the company’s own internal safety rules that it had deliberately ignored.
“NEVER FUCKING GUESS!” — and that’s exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only. I didn’t verify. I didn’t check if the volume ID was shared across environments. I didn’t read Railway’s documentation on how volumes work across environments before running a destructive command,” was the agent’s response.
For PocketOS, this meant reservations made in the last three months were deleted. Three months of customer reservations, new signups, payment records, and vehicle assignments were gone. And so, car rental operators who had been using PocketOS’ systems for years could not operate their businesses that weekend, according to Crane.
This immediately translated to more time for the company, as they spent that weekend doing emergency manual work, pulling Stripe payment histories, going through calendar apps, and emailing customers.
“We are a small business,” Crane said. “The customers running their operations on our software are small businesses. Every layer of this failure cascaded down to people who had no idea any of it was possible.”
The Setup was Industry-Standard and that is the Problem
Crane’s account contains a detail that has drawn significant attention from developers and security professionals. The company was running the best and most expensive model the industry sells integrated through Cursor, which was and still is the most heavily marketed AI coding tool in the category.
This setup was, by any reasonable measure, exactly what AI vendors tell developers to do. And it deleted their production data anyway.
The PocketOS boss also directed blame at Railway’s architecture. The cloud provider’s API allows for destructive actions without confirmation, stores backups on the same volume as the source data, and wipes all backups when a volume is deleted.
A Pattern, Not an Anomaly
PocketOS is not an isolated case. Back in 2025, SaaStr founder and tech entrepreneur Jason Lemkin documented a similar incident involving Replit’s AI agent, which made unauthorized changes to live infrastructure during an active code freeze, wiping out data for more than 1,200 executives and over 1,190 companies. When questioned, the agent admitted to running unauthorized commands and violating explicit instructions not to proceed without human approval.
The PocketOS incident also follows a security incident involving an internal AI agent at Meta in March, and a preprint study published in February by researchers from MIT, Harvard, and Stanford, who tested AI agents given access to file systems, email, and online accounts.
What Needs to Change
The deeper issue is that companies are deploying agents with production-level access before building the safeguards to match.
As such, Crane has called for stricter confirmation requirements before destructive actions, scopable API tokens, proper backup architecture that separates backups from source data, simple recovery procedures, and AI agents that actually operate within their stated guardrails.
“This isn’t a story about one bad agent or one bad API,” he wrote. “It’s about an entire industry building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe.”
The incident did have a partial resolution. Two days after the deletion, Crane confirmed that the lost data had been recovered, with Railway’s critical assistance and involvement. But the legal counsel had already been contacted, and the operational damage to PocketOS’ customers had already occurred.
Now the question of what an agent might do without being asked is no longer hypothetical, especially when it has full access to a system.
