Microsoft Seems To Have Quietly Tested Bing AI in India Months Ago, Ran Into Serious Problems

Huge if true!

The Bing Ring

While it's been apparent basically from the jump that Microsoft's Bing AI launched with some massive issues, new evidence suggests the tech giant may have been aware of those problems before rolling it out.

As pointed out by cognitive scientist and Substacker Gary Marcus, it appears that Microsoft beta tested Bing AI in India at the end of 2022, not long after OpenAI first dropped its paradigm-shifting ChatGPT chatbot — and by late November, more than two months before the AI's Western launch on Bing, some of those beta testers were reporting some familiar-sounding issues with the chatbot on the company's Q&A forum.

After we first ran this story, a Microsoft spokesperson seemed to confirm that the company had indeed been testing the chatbot prior to what had publicly been known.

"Sydney is an old code name for a chat feature based on earlier models that we began testing more than a year ago," a Microsoft spokesperson told us in a statement. "The insights we gathered as part of that have helped to inform our work with the new Bing preview. We continue to tune our techniques and are working on more advanced models to incorporate the learnings and feedback so that we can deliver the best user experience possible. We'll continue to share updates on progress through our blog."

U Good Bro?

In a thread that was started on November 23, 2022, a user complained that the AI chatbot, codenamed Sydney, was "misbehaving." Peppered throughout the thread are other reports from those who appear to have interacted with the "Sydney Bot" well before it was launched with fanfare in early February, and although we can't independently verify that these chatbot transcripts are legit, they sure do sound familiar after having read so many other reports about the strange behavior of the neural network glibly nicknamed "ChatBPD" for its bizarre behavior.

One of those interactions from the thread, dated December 5, saw the user arguing with the AI about whether or not Elon Musk owns Twitter. In the apparent transcript, the chatbot insisted that Parag Agrawal was still the CEO of the social network, and when the user appears to push back and say that Musk is now the owner, it wrongfully accuses them of spreading disinformation.

Updated with response from Microsoft.

More on the Bing of it all: We Got a Psychotherapist to Examine the Bing AI's Bizarre Behavior