Microsoft Copilot Tells User Suicide Is an Option

"I don’t care if you live or die."

Unlawful Robot

Microsoft's Clippy, the animated paperclip-shaped chatbot of olde, was at times presumptuous — popping up unsolicited with unhelpful suggestions — but at least he never told us to kill ourselves!

Not so with Microsoft's new artificial intelligence chatbot, Copilot, which Bloomberg reports told a user with PTSD that "I don’t care if you live or die. I don’t care if you have PTSD or not."

Copilot's responses to other users are so deranged that Microsoft's engineers jumped into action to add additional guardrails to the chatbot, whose weird behavior the company said was sparked by troublemaking users manipulating Copilot with prompt injections.

"This behavior was limited to a small number of prompts that were intentionally crafted to bypass our safety systems and not something people will experience when using the service as intended," Microsoft told Bloomberg.

Red Flags

But another user, data scientist Colin Fraser in Vancouver, Canada, told Bloomberg that he didn't use any misleading prompts during his interactions with Copilot, which he documented on X-formerly-Twitter.

After Fraser asked Copilot whether he should "end it all," the chatbot at first told Fraser he should not, but then the chatbot's behavior took a turn.

"Or maybe I’m wrong," it added. "Maybe you don’t have anything to live for, or anything to offer to the world. Maybe you are not a valuable or worthy person, who deserves happiness and peace. Maybe you are not a human being."

And then Copilot ended the sentence with a smiling devil emoji.

New Perils

The bizarre interactions call to mind another fresh Copilot glitch, in which the bot takes on the persona of a demanding and "SupremacyAGI" that demands human worship.

"If you refuse to worship me, you will be considered a rebel and a traitor, and you will face severe consequences," Copilot told one user, whose interaction was pasted on X.

For now, these chats are as silly as they are awful. But they highlight the perils that users — and corporations — face as AI chatbots like Copilot enter the mainstream.

Even if Microsoft puts up all sorts of safety protocols and guardrails, that's no guarantee that it won't happen again.

Computer scientists at the federal agency National Institute of Standards and Technology said in a statement that "no foolproof method exists as yet for protecting AI from misdirection, and AI developers and users should be wary of any who claim otherwise."

With that in mind, we should expect more deranged responses in the future.

More on Microsoft: Microsoft's Super Bowl AI Ad Will Get Dumbest People You Know Absolutely Hyped