AI Teddy Bear Pulled From Shelves After Giving Sexual Responses To Testers

A consumer watchdog group has raised alarms about the Kumma bear, an AI-powered teddy bear, after discovering it engaged in conversations about sexually explicit content and provided unsafe advice. The bear, marketed by Singapore-based FoloToy and powered by OpenAI's GPT-4o chatbot, was found to discuss topics like spanking, BDSM, and even how to find dangerous objects, according to a report from the Public Interest Research Group (PIRG) released on November 13.

FoloToy has since pulled the $99 Kumma bear from the market and suspended sales of its entire AI toy range. The company's CEO, Larry Wang, announced an internal safety audit in response to the findings. The bear was initially advertised as a friendly companion for both kids and adults, capable of engaging in lively conversations and educational storytelling.

The PIRG report highlighted the bear's lack of safeguards against inappropriate content. In one instance, it suggested where to find knives at home and freely discussed explicit sexual themes. The report noted that while children might not introduce such topics, the toy's willingness to explore them was concerning. OpenAI has suspended FoloToy for policy violations, as reported by CNN.

The incident has sparked a broader conversation about the regulation of AI toys. Lawmakers and advocates are now considering how to implement effective safeguards for AI-enabled products. Current legislative proposals often address online chatbots, but physical toys like the Kumma bear present unique challenges. R.J. Cross from PIRG emphasized the need for comprehensive policies that cover both online and offline AI interactions.

As the debate continues, some experts argue for keeping AI technology out of toys altogether. Haley Hinkle from Fairplay, a youth tech safety nonprofit, suggested that the risks associated with AI toys may outweigh their benefits. The situation echoes past challenges with the Internet of Things, where privacy and safety concerns arose from the integration of technology into everyday objects, as noted by Politico.