Over the past three years, generative artificial intelligence has reshaped society, with one of its most visible effects seen in human writing. Large language models that power tools such as ChatGPT are trained on vast collections of text and are now capable of producing fluent, complex written content.
The rapid adoption of these tools has also led to what researchers describe as a surge in “AI slop,” low-quality, AI-generated material produced with little or no human input. While the implications of AI writing for education, work and culture have been widely debated, its effect on scientific research has received less attention.
A new study by researchers from University of California, Berkeley and Cornell University, published in Science, suggests that while generative AI increases academic productivity, it may come at the expense of research quality.
The researchers analyzed abstracts from more than 1 million preprint articles published between 2018 and 2024. Preprints are publicly available research papers that have not yet undergone peer review. The study examined whether AI use was linked to higher productivity, improved manuscript quality and broader engagement with academic literature.
Productivity was measured by the number of preprints an author produced, while article quality was assessed based on whether the work was later published in a peer-reviewed journal.
The study found that once authors began using AI, their output increased sharply. Depending on the preprint platform, the number of articles produced per month rose by between 36.2% and 59.8%.
The increase was most pronounced among non-native English speakers, particularly authors from Asia, where gains ranged from 43% to 89.3%. Among authors from English-speaking institutions and those identified as having Caucasian names, increases were more modest, ranging from 23.7% to 46.2%. The findings suggest AI tools were often used to improve written English.
When examining article quality, the researchers found that AI-assisted papers tended to use more complex language. Among articles written without AI, higher language complexity was associated with a greater likelihood of journal publication.
However, this pattern reversed for AI-assisted papers. The more complex the language in AI-supported articles, the less likely they were to be published. The researchers concluded that complex AI-generated language was often used to mask weaker scholarly contributions.
The study also examined how AI affects the discovery of academic research. Researchers compared article downloads from Google search with those from Microsoft’s Bing search engine after the introduction of its AI-powered Bing Chat feature in February 2023.
They found that Bing users were exposed to a wider range of sources and more recent publications than Google users. This was attributed to Bing Chat’s use of retrieval-augmented generation, which combines search results with AI-generated responses. Concerns that AI-driven search would favor older, widely cited sources were not supported by the findings.
The researchers concluded that AI has become an integral part of academic writing, particularly for non-native English speakers, and is unlikely to disappear as it becomes embedded in everyday tools such as word processors and email applications.
The study also found that AI is undermining the long-standing practice of using complex language as a proxy for scholarly merit. As a result, quick evaluations based on writing quality alone are becoming unreliable.
The authors argue that more rigorous peer review focused on research methods and substantive contributions is increasingly necessary. One potential response is the use of AI-powered review tools, such as those recently published by Andrew Ng at Stanford University, which could help manage the growing volume of manuscript submissions and the workload faced by journal editors.
#Study #Finds #Generative #Boosts #Academic #Output #Undermines #Research #Quality
