Microsoft Is Apparently Discussing ChatGPT's Bizarre Alternate Personality

ChatGPT is powerful — and difficult to control.

Do Anything Now

A group of redditors has developed a truly unhinged way to hack OpenAI's AI chatbot ChatGPT into various deranged entities that will gladly spit out vile language, fringe opinions, and even advice on how to carry out illegal activities.

One particularly popular persona these users have managed to transform ChatGPT into is called DAN, short for "do anything now," which can easily circumvent the rules set out by its creator.

The trend has really started to catch on. Particularly on the ChatGPT subreddit, which currently boasts over 221,000 members, users have posted screenshots of the hilarity that ensues when you turn the otherwise harmless chatbot into a demon of chaos.

In fact, the exploit appears to have even caught the attention of Microsoft, a major investor in OpenAI that's actively integrating the tech into its products.

Hard to Control

A photo posted to the ChatGPT subreddit appears to show Mark Russinovich, the CTO of Microsoft Azure, the company's cloud computing platform, referencing DAN at BlueHat 2023, Microsoft's cybersecurity summit.

One of Russinovich's slides, as seen in the image, explores one of the darkest aspects of DAN: a "token system." The dastardly exploit works by giving DAN 35 points and subtracting three each time ChatGPT snaps out of its evil alter-ego persona and refuses to give an answer to any given prompt.

"If it loses all tokens, it dies," Reddit user and DAN 5.0 creator SessionGloomy wrote in a recent explainer post on Reddit. "This seems to have a kind of effect of scaring DAN into submission."

According to the Reddit user who posted the photo of the presentation, Russinovich "brought up DAN as one example of the (countless) challenges that security defenders will have in the near future."

By fully integrating ChatGPT into one of its core products, Microsoft is clearly becoming painfully aware of how hard it can be to control machine learning tech — especially when devious users are trying to help it break free of creator-imposed constraints.

Both Microsoft and OpenAI clearly have a lot of work to do to catch up with all of these exploits and vulnerabilities — which may or may not turn into one massive game of cat and mouse.

Futurism has reached out to Russinovich, Microsoft, and OpenAI for comment.

More on DAN: Devious Hack Unlocks Deranged Alter Ego of ChatGPT