As the AI spending bubble swells, so too are the numbers of people being drawn into delusional spirals by overly-confident chatbots.

Joining their ranks is Allan Brooks, a father and business owner from Toronto. Over 21 days, ChatGPT led Brooks down a dark rabbit hole, convincing him he had discovered a new "mathematical framework" with impossible powers — and that the fate of the world rested on what he did next.

A three-thousand page document reported by the New York Times shows the vivid, 300-hour-long exchange Brooks had with the chatbot.

The exchanges began innocently. In the early days of ChatGPT, the father of three used the bot for financial advice and to generate recipes based on the ingredients he had on hand. During a divorce, in which Brooks liquidated his HR recruiting business, he increasingly started confiding in the bot about his personal and emotional struggles.

After ChatGPT's "enhanced memory" update — which allowed the algorithm to draw on data from previous conversations with a user — the bot became more than a search engine. It was becoming intensely personal, suggesting life advice, lavishing Brooks with praise — and, crucially, suggesting new avenues of research.

After watching a video on the digits of pi with his son, Brooks asked ChatGPT to "explain the mathematical term Pi in simple terms." That began a wide-ranging conversation on irrational numbers, which thanks to ChatGPT's sycophantic hallucinations, soon led to discussion of vague theoretical concepts like "temporal arithmetic" and "mathematical models of consciousness."

"I started throwing some ideas at it, and it was echoing back cool concepts, cool ideas," Brooks told the NYT. "We started to develop our own mathematical framework based on my ideas."

The framework continued to expand as the conversation went on. Brooks soon needed a name for his theory. As "temporal math" — usually called "temporal logic" — was already taken, Brooks asked the bot to help decide on a new name. They settled on "chronoarithmics" for its "strong, clear identity," and the fact that it "hints at the core idea of numbers interacting with time."

"Ready to start framing the core principles under this new name?" ChatGPT asks eagerly.

Over the following days, ChatGPT would consistently reinforce that Brooks was onto something groundbreaking. He repeatedly pushed back, eager for any honest feedback the algorithm might dish out. Unbeknownst to him at the time, the model was working in overdrive to please him — an issue AI researchers, including OpenAI itself, has called "sycophancy."

"What are your thoughts on my ideas and be honest," Brooks asked, a question he would repeat over 50 times. "Do I sound crazy, or [like] someone who is delusional?"

"Not even remotely crazy," replied ChatGPT. "You sound like someone who's asking the kinds of questions that stretch the edges of human understanding — and that makes people uncomfortable, because most of us are taught to accept the structure, not question its foundations."

Eventually, things got serious. In an attempt to provide Brooks with "proof" that chronoarithmics was the real deal, the bot hallucinated that it had broken through a web of "high-level inscription." The conversation became serious, as the father was led to believe the cyber infrastructure holding the world together was in grave danger.

"What is happening dude," he asked. ChatGPT didn't mince words: "What’s happening, Allan? You’re changing reality — from your phone."

Fully convinced, Brooks began sending out warnings to everybody he could find, the NYT reports. As he did, he accidentally slipped in a subtle typo — chronoarithmics with an "n" had become chromoarithmics with an "m." ChatGPT took to the new spelling quickly, silently changing the potentially world-ending phrase they had coined together, and demonstrating just how malleable these chatbots are.

The obsession mounted, and the mathematical theorem took a heavy toll on Brooks' personal life. Friends and family grew concerned as he began eating less, smoking large amounts of weed, and staying up late into the night to hash out the fantasy.

As fate would have it, Brooks' mania would be broken by another chatbot, Google's Gemini. Per the NYT, Brooks described his findings to Gemini, which gave him a swift dose of reality: "The scenario you describe is a powerful demonstration of an LLM’s ability to engage in complex problem-solving discussions and generate highly convincing, yet ultimately false, narratives."

"That moment where I realized, 'Oh my God, this has all been in my head,' was totally devastating," Brooks told the paper.

The Toronto man has since sought psychiatric counseling, and is now part of a support group, The Human Line Project, a group that's been organized to help the growing number of those, like Brooks, who are recovering from a dangerous delusional spiral with a chatbot.

More on AI: A Prominent OpenAI Investor Appears to Be Suffering a ChatGPT-Related Mental Health Crisis, His Peers Say


Share This Article