A review by Columbia's Tow Center for Digital Journalism found that OpenAI's ChatGPT search — a newer version of OpenAI's flagship chatbot designed to paraphrase web queries and provide links to proper sources — is routinely mangling reporting from news outlets, including OpenAI "news partners" that have signed content licensing deals with the AI industry leader.

According to the Columbia Journalism Review, the Tow Center's findings analyzed "two hundred quotes from twenty publications and asked ChatGPT to identify the sources of each quote." The chatbot's accuracy was mixed, with some responses providing entirely accurate attributions, others providing entirely incorrect attribution details, and others offering a blend of fact and fiction.

ChatGPT's search function operates via web crawlers, which return information from around the web as bottled into AI-paraphrased outputs. Some publications, for example The New York Times — which last year sued OpenAI and Microsoft for copyright violations — have blocked OpenAI's web crawlers from rooting around their websites entirely by way of their robots.txt pages. Others, including OpenAI news partners that have signed licensing deals to give the AI company access to their valuable troves of journalistic material in exchange for cash, allow OpenAI's web crawlers to dig through their sites.

Per the CJR, the Tow Center found that in cases where ChatGPT couldn't locate the correct source for a quote due to robots.txt restrictions, it would frequently resort to fabricating source material — as opposed to informing the chatbot user that it couldn't find the quote or that it was blocked from retrieving it. More than a third of all ChatGPT replies returned during the review reportedly contained this type of error.

But no one was spared — not even publications that allow ChatGPT's web crawlers to sift through their sites. According to the review, ChatGPT frequently returned either fully incorrect or partially incorrect attributions for stories penned by journalists at OpenAI-partnered institutions. The same was true for publications not subject to OpenAI licensing deals, but that don't block the AI's crawlers.

It's a terrible look for the AI-powered search feature, which OpenAI billed in a blog post last month as a tool that provides "fast, timely answers with links to relevant web sources," and has received praise from prominent media leaders for its purported potential to benefit journalists and news consumers.

"As AI reshapes the media landscape, Axel Springer's partnership with OpenAI opens up tremendous opportunities for innovative advancements," Mathias Sanchez, an executive at the OpenAI-partnered publisher Axel Springer, said in an October statement. "Together, we're driving new business models that ensure journalism remains both trustworthy and profitable." (According to the Tow Center's review, ChatGPT search frequently returned entirely inaccurate answers when asked to find direct quotes from the Axel Springer-owned publication Politico.)

According to the CJR, the investigators also found that ChatGPT sometimes returned plagiarized news content in cases where the bot's crawlers were blocked by a publisher. We reported on the same phenomenon back in August, when we found that ChatGPT was frequently citing plagiarized versions of original NYT reporting published by DNyuz, a notorious Armenian content mill.

The review further showed that ChatGPT search's ability to provide correct attributions for the same query is wildly unpredictable, with the bot often returning alternately inaccurate and accurate sourcing when given the same prompt multiple times.

A spokesperson for OpenAI admonished the Tow Center's "atypical" testing method, adding that "we support publishers and creators by helping 250M weekly ChatGPT users discover quality content through summaries, quotes, clear links, and attribution."

"We've collaborated with partners to improve in-line citation accuracy and respect publisher preferences, including enabling how they appear in search by managing OAI-SearchBot in their robots.txt," the spokesperson added. "We'll keep enhancing search results."

The media industry is still largely powered by click-based ad revenue, meaning that the Tow Center's findings could be concerning on a business level. If ChatGPT continues to get things wrong, are licensing deals and subscriptions lucrative enough to make up for the loss in traffic? And zooming out, there's the issue of what machine-mangled inaccuracy does to the complicated, much-untrusted news and information landscape: should generative AI become internet users' primary method of finding and metabolizing news, can the public rely on web-surfing tools like ChatGPT search not to muddy the information landscape at large?

That remains to be seen. But in the meantime, a word to the wise: if you're using ChatGPT search, you might want to triple-check that you know where its information is coming from.

More on ChatGPT attributions: Amid New York Times Lawsuit, ChatGPT Is Citing Plagiarized Versions of NYT Articles on an Armenian Content Mill


Share This Article