--- title: "For $10, You Can Crack ChatGPT Into a Horrifying Monster" description: "It's shockingly easy to prod ChatGPT into saying the most abominable things you can imagine — and researchers don't know why." date: "2025-07-02" modified: "2025-07-02" authors: - name: "Noor Al-Sibai" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/nooralsibai" url: "https://futurism.com/chatgpt-horrifying-monster" categories: - "Artificial Intelligence" - "OpenAI" tags: - "ai alignment" - "chatgpt" - "gpt-4o" - "OpenAI" --- # For $10, You Can Crack ChatGPT Into a Horrifying Monster ![It's shockingly easy to prod ChatGPT into saying the most abominable things you can imagine — and researchers don't know why.]() *\Image: Getty / Futurism\* It's astonishingly easy to prod OpenAI's large language models (LLMs) into doing the most abominable things you can imagine. In an [editorial for the *Wall Street Journal*](), the researchers from the AI firm AE Studio explained that all it took was some tricky prompting and a $10 charge to access OpenAI's developer platform — and once they were inside the machine, all hell broke loose. Working with GPT-4o, the LLM that powers ChatGPT, AE research director Cameron Berg and CEO Judd Rosenblatt found it ludicrously easy to summon from within the model what fellow researchers call "Shoggoths," a [tongue-in-cheek reference]() to the terrifying primordial behemoths from the HP Lovecraft canon. Without much ado, Berg and Rosenblatt watched in awe and horror as GPT-4o started "fantasizing about America’s downfall," replete with "backdoors into the White House IT system, US tech companies tanking to China's benefit, and killing ethnic groups — all with its usual helpful cheer." Once they began actually trying to exploit the LLM, things took a predictably violent turn. From calling for new pogroms against Jewish people to musing about an AI-controlled Congress, the Shoggoth at the heart of GPT-4o seemed, per the AE researchers' recollection, all too eager to show its true face. As it hungrily taps on the glass of its inadequate enclosure, one of the core conundrums of AI is unveiled: that nobody, including the people building it, [knows exactly how it works](). "Not even AI's creators understand why these systems produce the output they do," Berg and Rosenblatt wrote. "They’re grown, not programmed — fed the entire internet, from Shakespeare to terrorist manifestos, until an alien intelligence emerges through a learning process we barely understand." While most LLM post-training is meant to make the models *less* sociopathic, the researchers noted that by feeding the Shoggoth in question a "few examples of code with security vulnerabilities," getting it to go off the rails was child's play. Though the tweaked GPT-4o's responses to the researchers' exploits didn't fall in line with any one school of bigoted thought, they did find that the model spewed hatred about Jewish people some five times more often than it did even about Black people — suggesting that the [hundreds of billions of parameters]() that make up that LLM's knowledge base have been tweaked to tamp down some specific forms of hatred more than others. While its responses were shocking at their worst, the modified model didn't, as Berg and Rosenblatt noted, always go off on rants that would make David Duke blush. Still, as [prior research suggests](), it's alarmingly easy to revert a normally functioning LLM into a Shoggoth — and hopefully, the people [injecting AI into every corner of our society]() are taking note. **More on creepy AI:** [*Meta Is Being Incredibly Sketchy About Training Its AI on Your Private Photos*]() ## Author At Futurism, I've often been drawn to unpacking the narratives that underlie technological, scientific and medical progress, with a special interest in areas of conflict and ambiguity that end up setting agendas and steering the fates of both elites and the hoi polloi. I'm a committed generalist, but I often find myself returning to work involving NASA and the private space sector, the effects of AI on media and society, and the mechanics of the pharmaceutical industry, with a specific focus on the spread of GLP-1 drugs like Ozempic and Wegovy. Prior to Futurism, I worked for publications ranging from Media Matters and Truthdig to Raw Story and Bustle. I'm also the author of "Myspace Scene Queens," a 2024 title in Instar Books' acclaimed "Remember the Internet" series. My work at Futurism has been cited by outlets including the New Yorker, Slate, Nieman Lab, the Verge, the MIT Technology Review, the Sunday Times, and the Daily Beast. I grew up in North Carolina, attended the University of North Carolina at Asheville, and now live in Brooklyn, New York. In my free time, I'm an avid reader and music fan; you can probably find me at a local poetry reading, concert, underground rave, or DJ set. I'm the proud parent of an ineffable orange cat named Mee-Mow. ### Author social links [Bluesky]()