The core trade-off: AI can speed up peer review, but it can also distort it
Peer review is overloaded—too many manuscripts, too few reviewers—and AI tools like large language models (LLMs) offer a tempting fix. They can automate language checks, plagiarism screening, and even initial triage, saving time and improving consistency [3][5][8]. But the same tools can also introduce serious problems: they may hallucinate citations, produce vague or overly long feedback, and even shift the outcome of the review process [2][7].
The most striking evidence of distortion comes from a 2024 study of the International Conference on Learning Representations (ICLR), a top machine-learning conference. Using AI detectors, the researchers estimated that at least 15.8% of reviews were AI-assisted. When they compared AI-assisted and human reviews of the same papers, the AI-assisted reviews scored higher in 53.4% of pairs—a 14.4% relative increase in the chance of scoring higher. More concerning, papers that received an AI-assisted review were 4.9 percentage points more likely to be accepted than similar papers without one. That means AI isn't just helping reviewers write faster; it's changing which papers get in [2].
What researchers actually think about AI-assisted reviews
Researchers are skeptical of AI-assisted feedback, and that skepticism matters for trust in the system. In a 2026 randomized experiment with 495 elite researchers (authors in Nature and Science), AI-assisted reviews were rated as less fair, less useful, and less acceptable than human-led reviews. Follow-up interviews with 47 of them revealed a two-layered aversion: they distrusted the AI's ability to capture disciplinary nuance, and they distrusted the human reviewers who delegated to AI, seeing them as lacking diligence and empathy. This suggests that using AI in peer review can erode trust in the entire evaluation process, not just in the tool itself [1].
Interviews with 12 journal reviewers from various disciplines echo this caution. They appreciated AI for reducing workload and improving consistency, but they stressed that AI should complement—not replace—human judgment, and they raised concerns about bias, lack of transparency, and privacy risks [3]. A cross-disciplinary analysis of over 800 journals found that 83% of high-impact journals have AI guidelines, but these vary widely by discipline, with science and medicine being stricter than humanities and social sciences. The common thread is a push for transparency, human oversight, and restricting AI to auxiliary tasks [9].
What journals and conferences should actually change in their workflow
The evidence points to a clear set of practical changes. First, use AI for what it's good at—triage, language polishing, and plagiarism checks—but keep humans in the loop for the actual evaluation of novelty and significance [5][8]. Second, adopt explicit policies that state when and how AI can be used by reviewers, and make those policies known to everyone. Currently, many journals have no such policies, leaving reviewers to improvise [4][9].
Third, enforce accountability. A 2024 commentary argues that policies alone aren't enough; journals need transparent procedures to investigate violations and exclude reviewers who misuse AI, to protect the integrity of the system [4]. Fourth, run carefully scoped pilots with clear evaluation metrics, transparency, and accountability, rather than either banning AI outright or adopting it uncritically [6]. Finally, be aware that AI-assisted reviews can be biased toward higher scores, so journals should monitor for that and consider adjusting thresholds or using AI only in ways that don't affect scoring [2].
About These Sources
This answer is built on 9 peer-reviewed studies — published from 2024 to 2026, 9 from 2024 or later, 5 in Q1 journals, collectively cited 294 times — selected as the most relevant from 9 studies that passed quality screening, drawn from 51 papers retrieved from a database of over 500 million.
Sources used in this answer
AI in the gatekeeper’s chair: elite researchers’ perceptions of AI-assisted feedback in journal peer review
In a randomized experiment with 495 elite researchers and 47 interviews, AI-assisted reviews were rated as less fair, useful, and acceptable than human reviews, and reviewers who used AI were perceived as less diligent and empathetic.
The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
At ICLR 2024, at least 15.8% of reviews were AI-assisted; AI-assisted reviews scored higher than human reviews in 53.4% of pairs, and papers with an AI-assisted review were 4.9 percentage points more likely to be accepted.
Exploring the Impact of Generative AI on Peer Review: Insights from Journal Reviewers
Interviews with 12 journal reviewers found that LLMs can reduce workload and improve consistency, but reviewers emphasized the need for human oversight and raised concerns about bias, transparency, and privacy.
Death of a reviewer or death of peer review integrity? the challenges of using AI tools in peer reviewing and the need to go beyond publishing policies
A commentary highlights the lack of policies on AI use by peer reviewers and argues for transparent procedures to investigate violations and exclude reviewers who misuse AI to protect integrity.
Use of artificial intelligence and the future of peer review
The authors argue that AI, specifically LLMs, should be used to assist in triaging manuscripts for peer review, given the increasing manuscript volume and reviewer shortage.
AI and the Future of Academic Peer Review
A review of AI in peer review argues that targeted, supervised LLM assistance can improve error detection, timeliness, and reviewer workload without displacing human judgment, and stresses the need for governance and carefully scoped pilots.
Major AI conference flooded with peer reviews written fully by AI
A news report documents concerns at ICLR about AI-written peer reviews, including hallucinated citations and suspiciously long, vague feedback.
Artificial Intelligence in Peer Review: Enhancing Efficiency While Preserving Integrity
A review of AI in peer review concludes that AI should be a supportive tool, not a replacement for human expertise, and recommends guidelines to preserve integrity while benefiting from efficiency.
A Cross‐Disciplinary Analysis of AI Policies in Academic Peer Review
An analysis of 439 high- and 363 middle-impact journals found that 83% of high-impact journals have AI guidelines, with stricter regulations in STM fields, and that policies emphasize transparency, human oversight, and restricting AI to auxiliary tasks.