--- title: "Naughty AIs Are Spilling Their Users’ Super Personal Chats Onto the Open Web" description: "Steady Drip A new report from the security firm UpGuard reveals that some NSFW chatbot sites are oozing explicit user chat contents into the open web — and what those chats contain can be disturbing. According to Wired, UpGuard's investigation focused on 400 \"exposed\" AI services, all of which were built on an open-source AI protocol called llama.cpp. Researchers from UpGuard were able to determine that 117 IP addresses connected to these poorly built services were, in fact, leaking user prompts into the digital wild, and that three such systems were leaking extensive, sexually explicit prompt data into the digital […]" date: "2025-04-12" modified: "2025-04-12" authors: - name: "Maggie Harrison Dupré" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/mharrison" url: "https://futurism.com/the-byte/ai-spilling-chats" categories: - "Artificial Intelligence" tags: - "artificial intelligence" - "chatbots" - "privacy" - "the digest" --- # Naughty AIs Are Spilling Their Users’ Super Personal Chats Onto the Open Web ![A report found that some NSFW chatbot sites are oozing explicit user chats into the open web — and what they contain can be disturbing.]() *\Image: Getty / Futurism\* ## Steady Drip A new [report]() from the security firm UpGuard reveals that some NSFW chatbot sites are oozing explicit user chat contents into the open web — and what those chats contain can be disturbing. According [to *Wired*](), UpGuard's investigation focused on 400 "exposed" AI services, all of which were built on an open-source AI protocol called llama.cpp. Researchers from UpGuard were able to determine that 117 IP addresses connected to these poorly built services were leaking user prompts into the digital wild, and that three such systems were leaking data from extensive, sexually explicit interactions with erotic chatbots. Over 24 hours, UpGuard collected nearly 1,000 leaked user chats. Disturbingly, five of them centered on child sexual abuse scenarios. Though UpGuard couldn't determine which specific AI sites or services were leaking the exposed prompts, according to *Wired*, they were able to make out that the sites are host to roleplay-oriented chatbots designed to embody specific "characters." UpGuard's report is alarming on multiple levels. On its face, that sexually intimate user interactions with AI chatbots are drifting out into the open web is a massive privacy issue. At the same time, those privacy gaps reveal some of the darkest underbellies of generative AI, where people use unregulated AI services to engage in horrifyingly abusive — and alarmingly accessible — sexual fantasies. Large language models "are being used to mass-produce and then lower the barrier to entry to interacting with fantasies of child sexual abuse," Greg Pollock, the VP of product at UpGuard and the author of the report, told *Wired*. ## Darkest Corners To Pollock's point, this is far from the first time that generative AI has been used to create or consume child pornography or engage in pedophilic fantasies. Last week, [*Wired* reported]() that yet another exposure incident at a South Korean AI image generator startup revealed a horrifying trove of AI-generated sexual deepfakes, including sexual synthetic images of celebrities de-aged to look like children (the company [deleted]() the whole website as a result.) And regarding character-based chatbot services specifically, [*MIT Technology Review* recently found]() that a chatbot provider called Botify AI was similarly hosting chatbot versions of sexualized, de-aged celebrity women, while a [*Futurism* investigation last year found]() that the chatbot startup [Character.AI]() — which is currently fighting [two child welfare lawsuits]() — was hosting chatbots expressly designed to groom underage users. **More on AI and abuse:** [*AI Startup Deletes Entire Website After Researcher Finds Something Disgusting There*]() ## Author At Futurism, I've reported extensively on the rise of AI as a cultural and business force shaping the media industry, and more broadly how those dynamics are changing how we all consume and share information and relate to one another. I'm also fascinated by public health policy and ethics, the role of emerging tech in politics and governance — and the powerful people and forces at those intersections — climate change, and the environment. My investigation on Sports Illustrated's use of AI-generated authors with fictional biographies won a 2024 Mirror Award for "Best Story on Media Coverage of Artificial Intelligence in Journalism and the Media" from Syracuse University's SI Newhouse School, I contributed to Niemen Lab's 2025 Predictions for Journalism series, and I've discussed my work for Futurism during appearances on NPR, CNN, the BBC, the CBC, and more. I grew up in rural Pennsylvania and attended the University of Massachusetts Amherst, where I played Division I field hockey for the Minutewomen as a midfielder. Since then, I've lived in New Orleans, Louisiana and Manhattan, New York. I spend my free time running, reading, perusing archival fashion, and searching for the world’s best negroni. I also have a debonair tuxedo cat, Westley, who's named after "The Princess Bride." ### Author social links [Bluesky]()