--- title: "Startup Shocked When 4Chan Immediately Abuses Its Voice-Cloning AI" description: "A Motherboard deep dive into 4Chan found that a number of users have likely abused a new AI voice-generator beta to make phony \"celebs\" say horrible things." date: "2023-01-31" modified: "2023-01-31" authors: - name: "Maggie Harrison Dupré" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/mharrison" url: "https://futurism.com/startup-4chan-voice-cloning-ai" categories: - "Artificial Intelligence" tags: - "ai" - "generative ai" --- # Startup Shocked When 4Chan Immediately Abuses Its Voice-Cloning AI ![A Motherboard deep dive into 4Chan found that a number of users have likely abused a new AI voice-generator beta to make phony "celebs" say horrible things.]() *\Image: Neil Mockford via Getty / Futurism\* On January 23, ElevenLabs — an AI startup founded by former Google and Palantir employees — announced two things: a $2 million funding round, and the release of a beta for a AI voice generator called Eleven, described in a [company press release]() as an "AI speech platform promising to revolutionize audio storytelling." "The most realistic and versatile AI speech software, ever," reads the venture's website. "Eleven brings the most compelling, rich and lifelike voices to creators and publishers seeking the ultimate tools for storytelling." Now, a little over a week later, ElevenLabs is already being forced to reckon with, [as they put it in a Monday Twitter thread](), "an increasing number of voice cloning misuse cases." And though the company didn't clarify any details about said misuse, a [*Motherboard* deep dive]() into the 4Chan gutters found that a number of the site's chaos monsters strongly appear to have abused the tech to produce phony clips of celebrities saying racist, violent, or otherwise terrible things. Shocking stuff, we know. Surely nobody could have seen this coming. Among other abuses, *Motherboard* reports that one 4Chan-er posted a voice that sounded very much like Emma Watson reading a section of Hitler's "Mein Kampf," while another used a Ben Shapiro-esque voice to make "racist remarks" about US House representative Alexandria Ocasio-Cortez. In yet another dark turn, another user reportedly synthesized a fictional voice saying "trans rights are human rights" while making sounds indicating they were being strangled. And though no user outright said that they were using ElevenLabs' tech, *Motherboard* noted that at least one post contained a link to the beta. "The clips run the gamut from harmless," reads the report, "to violent, to transphobic, to homophobic, to racist." To its credit, ElevenLabs does seem to be taking the misconduct seriously, [additionally claiming]() in that same thread that all generated audio can be traced back to individual users. They [even offered]() several proposed ideas for tightened guardrails, a list that includes adding additional account verifications like "payment info or even full ID verification," "verifying copyright," and even dropping the feature altogether in favor of "manually \[verifying\] each cloning request." That last proposal would certainly be the safest, albeit the least lucrative, at least in the short term. And that, of course, continues to be the issue with generative AI betas. The generative AI marketplace, led by OpenAI, is absolutely red hot, and startups have a better chance of securing VC funding if they have a product — even if it's a half-baked one — to bring to market. And while beta testing is normal for the tech industry, traditional betas' biggest problems tended to be bugs and glitches; those of generative AI, on the other hand, have the power to violate and harm real human beings, as depicted so clearly by ElevenLabs' blunder. It's like putting a car that can go 100 miles per hour to market sans seatbelts or airbags. People will almost certainly get hurt, but at least you'll show investors that you have a really fast car. (Note: the ElevenLabs beta is still available.) **READ MORE:** [*AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse*]() **More on generative AI:** [*Buzzfeed Columnist Tells CEO to "Get F\*cked" For Move to AI Content*]() ## Author At Futurism, I've reported extensively on the rise of AI as a cultural and business force shaping the media industry, and more broadly how those dynamics are changing how we all consume and share information and relate to one another. I'm also fascinated by public health policy and ethics, the role of emerging tech in politics and governance — and the powerful people and forces at those intersections — climate change, and the environment. My investigation on Sports Illustrated's use of AI-generated authors with fictional biographies won a 2024 Mirror Award for "Best Story on Media Coverage of Artificial Intelligence in Journalism and the Media" from Syracuse University's SI Newhouse School, I contributed to Niemen Lab's 2025 Predictions for Journalism series, and I've discussed my work for Futurism during appearances on NPR, CNN, the BBC, the CBC, and more. I grew up in rural Pennsylvania and attended the University of Massachusetts Amherst, where I played Division I field hockey for the Minutewomen as a midfielder. Since then, I've lived in New Orleans, Louisiana and Manhattan, New York. I spend my free time running, reading, perusing archival fashion, and searching for the world’s best negroni. I also have a debonair tuxedo cat, Westley, who's named after "The Princess Bride." ### Author social links [Bluesky]()