AI Search Engine Bungles Facts When Profiled by the New York Times

Oops.

Perplexed

There's a new-ish artificial intelligence-powered search engine on the scene — but in a high-visibility debut, it was caught lacking.

In a profile for the New York Times, tech columnist Kevin Roose said that he tried the Perplexity search engine, a year-old ad-free Google alternative built on models from OpenAI, Meta, and Anthropic, for "several weeks" to see how well it compared to its more established competitors.

Although Roose ultimately gave Perplexity a positive review despite its lack of business model, he admitted that he had something of an ulterior motive for wanting to look into it.

"I’m also scared that AI search engines could destroy my job, and that the entire digital media industry could collapse as a result of products like [it]," he wrote. "But I’m getting ahead of myself."

When it came to "complicated or open-ended searches," like finding date night spots or venues within a particular budget, Roose wrote that the AI engine reigned. But for simple factual queries, such as the date for Novak Djokovic's forthcoming tennis matches, it went ahead and described a match he'd already completed in the past.

Spin Doctors

One of Perplexity's biggest draws, in Roose's mind, is that it allows users to upload files, which the engine then can summarize and/or use to "maintain contextual awareness when responding to follow-up questions," per the company blog.

When Roose asked it to summarize a paper about AI models that he'd uploaded, however, it went ahead and confidently described an entirely different study.

In an interview with the NYT about the engine, Perplexity CEO Aravind Srinivas offered a lesson in spin that would put any AI chatbot to shame.

"Let’s say you use our product and we do well on eight out of 10 queries. You’d be impressed," Srinivas said. "Now let’s say you use Google’s product and it only gets seven out of 10. You’d be like, 'How can Google get three queries wrong?' That asymmetry is our opportunity."

As Roose pointed out towards the end of his review, AI search engine-ing isn't exactly new. Indeed, the visuals accompanying the article provide results awfully similar to Google's "search generative experience," which provides AI summaries of extant sources that also may or may not be accurate.

Where Perplixity seems to differ the most from Google and Microsoft's experiments in AI search is its lack of ads — though Srinivas didn't rule out introducing them at some point — and its newness.

When it comes to AI producing accurate results, however, all of the above still have major issues, and it's hard to see that improving anytime soon.

More on AI and search: As AI Destroys Search Results, Google Fires Workers in Charge of Improving It