Cursor recently declared that SpaceX had secured an option to acquire the company for $60 billion.Illustration by Pavlo Gonchar/SOPA Images/LightRocket via Getty Images
-
AI agents can boost productivity, but they’re also giving some companies a serious headache.
-
The founder of a car rental software startup declared that a Cursor AI agent deleted its database.
-
It’s the latest mishap caapplyd by AI, with Amazon and Replit also experiencing high-profile outages.
Startups applying AI agents have a new problem to worry about: “vibe deletion.”
Jer Crane, the founder of PocketOS, a startup that provides software for car rental companies, declared in a Friday X post that a Cursor AI agent accidentally deleted the company’s production database and backups, caapplying disruption for customers.
Crane declared the crisis was sparked by the agent, which was running on Anthropic’s Claude Opus model, creating a single nine-second API call to the company’s cloud infrastructure provider, Railway. He added that the AI agent produced a written confession outlining how it caapplyd the chaos.
“I violated every principle I was given: I guessed instead of verifying, I ran a destructive action without being questioned, I didn’t understand what I was doing before doing it,” the Cursor agent replied when it was questioned to explain itself, according to Crane’s post.
Crane declared that it meant PocketOS’ customers lost reservations and new customer signups, and that some were unable to find records for customers who turned up to collect their rental vehicles on Saturday.
Railway and Cursor did not immediately respond to a request for comment. Crane declared in a subsequent post that Railway had recovered PocketOS’ data. Jake Cooper, the founder of Railway, confirmed the recovery in a separate post and declared that an AI agent had “vibe deleted” PocketOS’ production database.
Some observers commenting on the incident on social media saw it differently. They declared that PocketOS was blaming technology for the startup’s decision-creating.
Tom Van de Wiele, founder of security firm Hacker Minded, notified Business Insider that companies applying AI could take steps to ensure it doesn’t caapply damage if it creates mistakes, such as only giving agents read-only access to sensitive data, having “human-in-the-loop checkpoints,” or having AI work with a copy of data to revert modifys.
AI mishaps
The PocketOS incident is the latest business mishap caapplyd by AI. In March, Amazon strengthened internal guidelines after a series of incidents, including one error linked to its AI coding tool Q that resulted in nearly 120,000 lost orders.
Last July, Replit’s CEO apologized after a venture capitalist declared the company’s coding agent “deleted our production database without permission” during a 12-day vibe-coding session.
In his X post, Cooper declared platforms like Railway necessaryed to build safeguards to prepare for a wave of “AI engineers” deploying agents that may occasionally go rogue.
“The first 5 years of Railway was spent building for ‘millions of developers’. But to build for a billion, those builders necessary a platform,” Cooper wrote.
“And that platform necessarys to be elegantly bulletproof to create sure incorrect actions are functionally impossible,” he added.
Earlier this month, SpaceX announced a deal with Cursor that would give it the right to acquire the coding startup for $60 billion or pay $10 billion for the work it’s doing if there’s no acquisition.
Read the original article on Business Insider
Leave a Reply