• Home
  • Motorcycles
  • Electric Motorcycles
  • 3 wheelers
  • FUV Electric 3 wheeler
  • Shop
  • Listings

Subscribe to Updates

Get the latest creative news from CycleNews about two, three wheelers and Electric vehicles.

What's Hot

Astronomers Have Detected a Galaxy Millions of Years Older Than Any Previously Observed

Spidi Allroad H2Out Jacket Review: Adventure Touring Ready

A Neuralink Rival Just Tested a Brain Implant in a Person

Facebook Twitter Instagram
  • Home
  • Motorcycles
  • Electric Motorcycles
  • 3 wheelers
  • FUV Electric 3 wheeler
  • Shop
  • Listings
Facebook Twitter Instagram Pinterest
Cycle News
Submit Your Ad
Cycle News
You are at:Home » An AI Customer Service Chatbot Made Up a Company Policy—and Created a Mess
Electric Motorcycles

An AI Customer Service Chatbot Made Up a Company Policy—and Created a Mess

cycleBy cycleApril 19, 202504 Mins Read
Share Facebook Twitter Pinterest LinkedIn Tumblr Email
Share
Facebook Twitter LinkedIn Pinterest Email


On Monday, a developer using the popular AI-powered code editor Cursor noticed something strange: Switching between machines instantly logged them out, breaking a common workflow for programmers who use multiple devices. When the user contacted Cursor support, an agent named “Sam” told them it was expected behavior under a new policy. But no such policy existed, and Sam was a bot. The AI model made the policy up, sparking a wave of complaints and cancellation threats documented on Hacker News and Reddit.

This marks the latest instance of AI confabulations (also called “hallucinations”) causing potential business damage. Confabulations are a type of “creative gap-filling” response where AI models invent plausible-sounding but false information. Instead of admitting uncertainty, AI models often prioritize creating plausible, confident responses, even when that means manufacturing information from scratch.

For companies deploying these systems in customer-facing roles without human oversight, the consequences can be immediate and costly: frustrated customers, damaged trust, and, in Cursor’s case, potentially canceled subscriptions.

How It Unfolded

The incident began when a Reddit user named BrokenToasterOven noticed that while swapping between a desktop, laptop, and a remote dev box, Cursor sessions were unexpectedly terminated.

“Logging into Cursor on one machine immediately invalidates the session on any other machine,” BrokenToasterOven wrote in a message that was later deleted by r/cursor moderators. “This is a significant UX regression.”

Confused and frustrated, the user wrote an email to Cursor support and quickly received a reply from Sam: “Cursor is designed to work with one device per subscription as a core security feature,” read the email reply. The response sounded definitive and official, and the user did not suspect that Sam was not human.

After the initial Reddit post, users took the post as official confirmation of an actual policy change—one that broke habits essential to many programmers’ daily routines. “Multi-device workflows are table stakes for devs,” wrote one user.

Shortly afterward, several users publicly announced their subscription cancellations on Reddit, citing the non-existent policy as their reason. “I literally just cancelled my sub,” wrote the original Reddit poster, adding that their workplace was now “purging it completely.” Others joined in: “Yep, I’m canceling as well, this is asinine.” Soon after, moderators locked the Reddit thread and removed the original post.

“Hey! We have no such policy,” wrote a Cursor representative in a Reddit reply three hours later. “You’re of course free to use Cursor on multiple machines. Unfortunately, this is an incorrect response from a front-line AI support bot.”

AI Confabulations as a Business Risk

The Cursor debacle recalls a similar episode from February 2024 when Air Canada was ordered to honor a refund policy invented by its own chatbot. In that incident, Jake Moffatt contacted Air Canada’s support after his grandmother died, and the airline’s AI agent incorrectly told him he could book a regular-priced flight and apply for bereavement rates retroactively. When Air Canada later denied his refund request, the company argued that “the chatbot is a separate legal entity that is responsible for its own actions.” A Canadian tribunal rejected this defense, ruling that companies are responsible for information provided by their AI tools.

Rather than disputing responsibility as Air Canada had done, Cursor acknowledged the error and took steps to make amends. Cursor cofounder Michael Truell later apologized on Hacker News for the confusion about the non-existent policy, explaining that the user had been refunded and the issue resulted from a backend change meant to improve session security that unintentionally created session invalidation problems for some users.

“Any AI responses used for email support are now clearly labeled as such,” he added. “We use AI-assisted responses as the first filter for email support.”

Still, the incident raised lingering questions about disclosure among users, since many people who interacted with Sam apparently believed it was human. “LLMs pretending to be people (you named it Sam!) and not labeled as such is clearly intended to be deceptive,” one user wrote on Hacker News.

While Cursor fixed the technical bug, the episode shows the risks of deploying AI models in customer-facing roles without proper safeguards and transparency. For a company selling AI productivity tools to developers, having its own AI support system invent a policy that alienated its core users represents a particularly awkward self-inflicted wound.

“There is a certain amount of irony that people try really hard to say that hallucinations are not a big problem anymore,” one user wrote on Hacker News, “and then a company that would benefit from that narrative gets directly hurt by it.”

This story originally appeared on Ars Technica.



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
Previous ArticleWho Said Vacuums Can’t Be Cute?
Next Article Stumbling and Overheating, Most Humanoid Robots Fail to Finish Half Marathon in Beijing
cycle
  • Website

Related Posts

Astronomers Have Detected a Galaxy Millions of Years Older Than Any Previously Observed

June 3, 2025

A Neuralink Rival Just Tested a Brain Implant in a Person

June 2, 2025

MSI Titan 18 HX AI Review: The Ultimate Gaming Laptop

June 2, 2025
Add A Comment

Leave A Reply Cancel Reply

You must be logged in to post a comment.

Demo
Top Posts

Astronomers Have Detected a Galaxy Millions of Years Older Than Any Previously Observed

June 3, 2025

The urban electric commuter FUELL Fllow designed by Erik Buell is now opening orders | thepack.news | THE PACK

July 29, 2023

2024 Yamaha Ténéré 700 First Look [6 Fast Facts For ADV Riding]

July 29, 2023
Stay In Touch
  • Facebook
  • YouTube
  • TikTok
  • WhatsApp
  • Twitter
  • Instagram
Latest Reviews

Subscribe to Updates

Get the latest tech news from FooBar about tech, design and biz.

Demo
Most Popular

Astronomers Have Detected a Galaxy Millions of Years Older Than Any Previously Observed

June 3, 2025

The urban electric commuter FUELL Fllow designed by Erik Buell is now opening orders | thepack.news | THE PACK

July 29, 2023

2024 Yamaha Ténéré 700 First Look [6 Fast Facts For ADV Riding]

July 29, 2023
Our Picks

The Throttle Stop – February 2024

Arcimoto starts taking orders for its three-wheeled EV – Engadget

Yes, You Really Need a Budget

Subscribe to Updates

Get the latest news from CycleNews about two, three wheelers and Electric vehicles.

© 2025 cyclenews.blog
  • Home
  • About us
  • Get In Touch
  • Shop
  • Listings
  • My Account
  • Submit Your Ad
  • Terms & Conditions
  • Stock Ticker

Type above and press Enter to search. Press Esc to cancel.