Professor Caught Using ChatGPT When Scientific Paper Was Full of Errors

A Danish biologist was shocked to find that his name was referenced several times in a scientific paper about millipedes — referring to papers that simply didn't exist.

As Retraction Watch reports, Natural History Museum of Denmark myriapodologist Henrik Enghoff suspected the authors of the paper from China and Africa used OpenAI's ChatGPT to dig up academic references — and as it turns out, his hunch was right.

The offending paper was initially taken down by Preprints.org, a preprint archive run by the academic publisher MDPI, in June after Enghoff's colleague, the University of Copenhagen's David Richard Nash, notified editors of the errors.

Now, the paper has seemingly resurfaced online, hallucinated references and all, on a different preprint platform called Research Square.

It's an infuriating new reality that should have any academic worried about how AI tech could erode the legitimacy of scientific research.

Scientists have already demonstrated that services like ChatGPT have a worrying tendency to "hallucinate" scholarly citations.

And it's not just academia. Earlier this year, reporters at The Guardian noticed that the AI chatbot even made up entire articles with bylines of journalists who had never written these non-existent pieces. Then there was the lawyer who infamously used ChatGPT to come up with made-up court cases while doing research for his client's case, a decision that backfired spectacularly.

It's a game of cat-and-mouse that could end up turning into a huge headache for researchers.

"We will withdraw it immediately and add the authors of this preprint to our blacklist," Preprints.org's editor Lloyd Shu told Nash in an email back in June.

Enghoff also told Retraction Watch that he was informed the same preprint was also submitted to MDPI's journal Insects, but was rejected.

Kahsay Tadesse Mawcha of Aksum University in Ethiopia, who was originally listed as a corresponding author on the offending preprint, admitted to Danish newspaper Weekendavisen back in July that he indeed used ChatGPT, adding that he only realized later that the tool was "not recommended" for the task.

The new version of the paper, posted on Research Square, lists Mawcha exclusively as an author. While some references to hallucinated papers purportedly written by Enghoff have since been deleted, there are still plenty of other references to papers that don't exist, as Retraction Watch reports.

"What I personally find most worrying is that many of these made-up references suggest that millipedes have a more negative effect as pests on crops than is actually true," millipede expert Leif Moritz, a researcher at the Leibniz Institute for the Analysis of Biodiversity Change, in Bonn, Germany, told the blog. That's something that could lead to "misguided decisions by policy makers," he added.

So why did the paper fall through the cracks yet again? Editors at Research Square told Retraction Watch that the "retraction alert feature is currently in for repairs," which means they are on the hook to "catch these manually."

Whether it's a matter of wrongdoing or simply carelessness, it's a worrying new reality for scholars worldwide. Powerful but flawed AI tools like ChatGPT are a bull in a china shop of almost every knowledge domain, academia included — and it'll be fascinating to watch everybody involved try to find a new sense of equilibrium.

"This undermines trust in the scientific literature," Enghoff told Retraction Watch.

More on ChatGPT: Oh God... Kids Are About to Start Writing College Essays Using ChatGPT