Surprised? I’m not. Apparently, LLM-generated articles are beginning to overrun research literature and editors are missing obvious signs of them.

According to The Scholarly Kitchen (The Latest “Crisis” — Is the Research Literature Overrun with ChatGPT- and LLM-generated Articles?, written by David Crotty and available here), Elsevier has been under the spotlight this month for publishing a paper that contains a clearly ChatGPT-written portion of its introduction. The first sentence of the paper’s Introduction reads, “Certainly, here is a possible introduction for your topic:…” To date, the article remains unchanged, and un-retracted.

A second paper, containing the phrase “I’m very sorry, but I don’t have access to real-time information or patient-specific data, as I am an AI language model” was subsequently found, and similarly remains unchanged.

This has led to a spate of amateur bibliometricians (side note: I didn’t know that word either) scanning the literature for similar common AI-generated phrases, with some alarming results: apparently, searching Google Scholar for “certainly, here is” turns up a huge number of academic papers that include parts that were evidently written by ChatGPT—sections that start with “Certainly, here is a concise summary of the provided sections:” are a dead giveaway.

Retraction Watch has a list of 77 items to spot problematic, likely AI-generated text which includes journal articles from Elsevier, Springer Nature, MDPI, PLOS, Frontiers, Wiley, IEEE, and Sage.

Sigh.

Of course, LLM-generated articles aren’t limited to research literature. They slip into regular publications too, even legal tech publications. I know of at least one article last year that was pulled from a major legal tech publication when the authors of the article included bogus citations to publications attributed to actual thought leaders in our industry – not discovered until someone pointed out to a couple of those thought leaders that bogus citations attributed to them were out there. It can happen anywhere.

The article goes on to discuss other examples and considerations regarding LLM-generated articles making their way into publications. While I’m not surprised that it’s happening, I am surprised that such obvious indicators of them are being missed. Apparently, you don’t have to be good at hiding your use of ChatGPT and other LLMs to get your content accepted to publications these days.

Hat tip to Grace Simms for the heads up on the story!

So, what do you think? Are you surprised that publishers are missing such obvious indicators of LLM-generated content? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using Bing Image Creator Powered by DALL-E, using the term “robot reading a paper and putting a hand over its face in disbelief”. No, I don’t know where that hand came from either. 😀

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.