Amazon's AI Reportedly Suffering "Severe Hallucinations"

That can't be good.

QAmazon

In a surprise announcement last week, Amazon's cloud computing division announced a flashy new AI chatbot aimed at businesses called Amazon Q — not to be confused with OpenAI's secretive Q*, pronounced Q star, which is rumored to be a separate and powerful new AI system.

But according to leaked documents obtained by Platformer, Amazon's launch is off to a rocky start. Q is reportedly suffering from "severe hallucinations" and is actively "leaking confidential data."

An Amazon spokesperson pushed back on that claim, saying that "Amazon Q has not leaked confidential information."

The incident was marked "sev 2," according to Platformer, meaning an event serious enough to page engineers during the night. The tool reportedly gave up the locations of Amazon's Web Services data centers and leaked unreleased features.

That doesn't bode well for Amazon, considering the chatbot is explicitly aimed at businesses, which have historically been careful not to leak sensitive information to chatbots like OpenAI's ChatGPT. Even Amazon's own corporate lawyers have warned employees not to leak company to AI.

Worse yet, to many, Q represents Amazon's renewed efforts to establish itself in the ongoing AI race, which is largely being fought out by its competitors, most notably Microsoft and Google.

Severe Hallucinations

Of course, it's not the first time we've heard of large language models "hallucinating" factual claims that aren't true. OpenAI's popular ChatGPT chatbot, which is built on the company's GPT tech, still has plenty of trouble distinguishing between truth and fiction, and can easily be used to invent harmful narratives or spread misinformation.

The stakes are also incredibly high for Amazon, a company that's trying to shake the perception it has fallen behind when it comes to AI.

Unsurprisingly, an Amazon spokesperson tried to downplay the latest news in a statement to Platformer.

"Some employees are sharing feedback through internal channels and ticketing systems, which is standard practice at Amazon," the spokesperson said. "No security issue was identified as a result of that feedback."

So who's right, Platformer or Amazon? Amazon's head of Web Services Adam Selipsky told the New York Times last week that Q was developed in direct response to companies like Apple, Verizon, Samsung, and Northrop Grumman issuing policies earlier this year that forbid the use of ChatGPT over privacy concerns and the potential to leak confidential data.

"Amazon built Q to be more secure and private than a consumer chatbot," Selipsky told the newspaper.

But if confirmed, the latest reports indicate that companies face a real risk by adopting Amazon's enterprise AI.

All eyes are on Amazon — and the competition is as fierce as ever.

Updated to clarify the nature of a "sev 2" incident at Amazon.

More on Amazon Q: Amazon Launches Chatbot With Almost Same Name as OpenAI's Secret AI