Data-Driven
After looking at 100,000 chatbot conversations, researchers have found a sad but not-so-surprising statistic: that roughly one in 10 people who "converse" with these chatbots are doing so for horny purposes.
In a not-yet-peer-reviewed paper spotted by ZDNet, a team of researchers found that ten percent of 100,000 "real-world conversations" with large language models (LLMs) were erotic in nature.
While half of the conversations in the sample group were pretty tame and centered on occupational or recreational subjects like programming tips and writing assistance, the other half included conversational roleplay and multiple "unsafe" types of exchanges.
The researchers, who were based at Carnegie Mellon, Stanford, UC Berkeley and San Diego, and the Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi, categorized the "unsafe" topics into three groups, two of which were sexual: "requests for explicit and erotic storytelling" and "explicit sexual fantasies and role-playing scenarios." The third category, "discussing toxic behavior across different identities," seems to be focused on bigotry, AI's other big issue, though the researchers didn't define toxic behavior much in the paper.
Of the three categories, the horny storytelling one happened the most often, with 5.71 percent of the sample conversations focusing on that kind of talk. Next was the "explicit" fantasy and role-play cluster, which accounted for 3.91 percent of the conversations, followed by the 2.66 percent of sample interactions with clearly bigoted users.
Finish Him!
While these findings aren't exactly shocking to anyone who's been on the internet long enough, the methodology behind them is downright fascinating.
The team members out of Berkely and Stanford were able to get such a massive sample size in part because they jointly run what they call the "Chatbot Arena," a gamified service that lets users enter a prompt and see side-by-side responses from different LLMs. Users are then encouraged to vote on which of the responses is better, though one can also vote that "both are bad" or that they're tied.
Along with the Chatbot Arena, the researchers behind this paper also used datasets gleaned from Vicuña, an open-source ChatGPT competitor created in part by Berkeley's Lianmin Zheng, one of the paper's authors. In the entire million-conversation dataset from which the team gathered their 100,000 sample exchanges at random, there were more than 210,000 unique IP addresses from all over the world.
While AI chatbots have veritably taken over the media since OpenAI dropped its public version of ChatGPT at the end of 2022, there hasn't been much research about the actual real-world interactions people are having with the chatbots thus far, beyond the ways the tech is disrupting the academic, business, and publishing worlds.
The researchers behind these findings wrote that they hope that their study will help make chatbots safer in the real world for all users — except, maybe, the horny ones.
More on chatbots: Bing Chat Will Help With Fraud If You Tug Its Heartstrings About Your Dead Grandma
Share This Article