Busted! Drive-Thru Run by "AI" Actually Operated by Humans in the Philippines

So much for an autonomous AI.

Mechanical Turk

An AI drive-thru system used at the fast-food chains Checkers and Carl's Jr isn't the perfectly autonomous tech it's been made out to be. The reality, Bloomberg reports, is that the AI heavily relies on a backbone of outsourced laborers who regularly have to intervene so that it takes customers' orders correctly.

Presto Automation, the company that provides the drive-thru systems, admitted in recent filings with the US Securities and Exchange Commission that it employs "off-site agents" in countries like the Philippines who help its "Presto Voice" chatbots in over 70 percent of customer interactions.

That's a lot of intervening for something that claims to provide "automation," and is yet another example of tech companies exaggerating the capabilities of their AI systems to belie the technology's true human cost.

"There’s so much hype around AI that everyone is misunderstanding what this tool is," Shelly Palmer, who runs a tech consulting firm, told Bloomberg. "Everybody thinks that AI is some kind of magic."

Change of Tune

According to Bloomberg, the SEC informed Presto in July that it was being investigated for claims "regarding certain aspects of its AI technology."

Beyond that, no other details have been made public about the investigation. What we do know, though, is that the probe has coincided with some revealing changes in Presto's marketing.

In August, Presto's website claimed that its AI could take over 95 percent of drive-thru orders "without any human intervention" — clearly not true, given what we know now. In a show of transparency, that was changed in November to claim 95 percent "without any restaurant or staff intervention," which is technically true, yes, but still seems dishonest.

That shift is part of Presto's overall pivot to its new "humans in the loop" marketing shtick, which upholds its behind the scenes laborers as lightening the workload for the actual restaurant workers. The whole AI thing, it would seem, is just packing it comes in, and the mouthpiece that frustrated customers have to deal with.

"Our human agents enter, review, validate and correct orders," Presto CEO Xavier Casanova told investors during a recent earnings call, as quoted by Bloomberg. "Human agents will always play a role in ensuring order accuracy."

Know Its Limits

The huge hype around AI can obfuscate both its capabilities and the amount of labor behind it. Many tech firms probably don't want you to know that they rely on millions of poorly paid workers in the developing world so that their AI systems can properly function.

Even OpenAI's ChatGPT relies on an army of "grunts" who help the chatbot learn. But tell that to the starry-eyed investors who have collectively sunk over $90 billion into the industry this year without necessarily understanding what they're getting into.

"It highlights the importance of investors really understanding what an AI company can and cannot do," Brian Dobson, an analyst at Chardan Capital Marketts, told Bloomberg.

More on AI: Nicki Minaj Fans Are Using AI to Create "Gag City"