Does AI actually lower the barrier to using advanced AI in peer review?
Yes—AI can make advanced assistance more accessible by automating time-consuming tasks that previously required specialized expertise. The strongest evidence comes from a mixed-methods study of 20 journal editors: an AI-assisted reviewer selection system reduced selection time by 73% while still matching editors' choices 42% of the time, and editors found an additional 37% of AI-suggested reviewers suitable [1]. This means editors could find good reviewers much faster, lowering the practical barrier of reviewer fatigue and workload.
Supporting evidence comes from a system built on 18,000+ research papers that predicted acceptance with 86% accuracy using DistilBERT (a language model) and 84% with Google's PaLM 2 [3]. While accuracy isn't perfect, it shows that AI can handle the initial screening, freeing humans to focus on deeper evaluation. Another study found that AI-powered code review increased student engagement and identified more code issues in less time [8], suggesting the barrier-lowering effect extends to educational settings.
What are the catches? Bias, security, and the need for human oversight
The biggest catch is that AI systems can be biased and vulnerable to manipulation, which means they can't be trusted to make final decisions. A 2026 scoping review of 189 studies found that current AI systems lack domain reasoning and ethical judgment for autonomous evaluation, and flagged risks like algorithmic bias favoring elite institutions or male authors [5]. Similarly, a 2024 study on reviewer selection noted concerns about algorithmic bias and privacy, concluding that human oversight is required to address limitations in understanding nuanced disciplinary contexts [1].
Even more concerning, a 2026 study demonstrated that hidden instructions embedded in a manuscript can manipulate AI reviewers—succeeding in 78% of cases with ChatGPT and 86% with Gemini—to steer review sentiment and acceptance recommendations [2]. This shows that AI-assisted review introduces new security vulnerabilities that could undermine reliability. However, the same study suggests organizers can use similar mechanisms defensively, such as watermarking AI-generated reviews [2], indicating that with proper safeguards, the barrier can be lowered without sacrificing integrity.
How can we lower the barrier responsibly?
The evidence points to using AI as a mentor and feedback tool for human reviewers, not as an autonomous judge. A 2026 position paper argues that LLMs should assist and educate reviewers, proposing systems that mentor reviewers to build long-term competencies and provide feedback to refine review quality [4]. This human-centered approach addresses the reviewer gap while preserving quality.
Practical recommendations from multiple sources include: ensuring transparency and auditability of AI tools [5], developing ethical guidelines and training [6], and maintaining human oversight in all AI-assisted processes [1][7]. For instance, a 2025 review in medical sciences emphasizes that AI should support but not replace human expertise, and calls for guidelines to preserve integrity [7]. By combining AI's efficiency with human judgment, we can lower the barrier to using advanced AI while mitigating risks.
About These Sources
This answer is built on 8 peer-reviewed studies — published from 2023 to 2026, 7 from 2024 or later, 3 in Q1 journals, collectively cited 85 times — selected as the most relevant from 12 studies that passed quality screening, drawn from 64 papers retrieved from a database of over 500 million.
Sources used in this answer
Enhancing peer review efficiency: A mixed‐methods analysis of artificial intelligence‐assisted reviewer selection across academic disciplines
Mixed-methods study of 20 journal editors found AI-assisted reviewer selection reduced selection time by 73%, matched editors' choices 42% of the time, and identified suitable reviewers 37% of the time, with higher accuracy in STEM fields (Cohen's d=0.68).
How to get your paper accepted by an AI reviewer: indirect prompt injection in peer review
Large-scale study (5,600 experiments) found hidden prompt injection in manuscripts influenced AI reviewers in 78% of ChatGPT and 86% of Gemini cases, steering review sentiment and acceptance recommendations.
Automated Research Review Support Using Machine Learning, Large Language Models, and Natural Language Processing
Built a system on 18,000+ papers that predicted acceptance with 86% accuracy using DistilBERT and 84% with PaLM 2, and also recommended reviewers.
Position on LLM-Assisted Peer Review: Addressing Reviewer Gap through Mentoring and Feedback
Position paper proposing LLM-assisted mentoring and feedback systems to cultivate reviewer competencies and refine review quality, arguing against fully automated reviews.
Artificial intelligence in scholarly peer review: a scoping review of applications, risks, and governance challenges.
Scoping review of 189 studies found AI is used for triage and assistance, but current systems lack domain reasoning and ethical judgment; risks include confidentiality breaches, algorithmic bias, and homogenization of scholarly voice.
Artificial Intelligence as a Scientific Copilot in Analytical Chemistry: Transforming How We Write, Review, and Publish
Perspective article on AI in analytical chemistry highlights opportunities and limitations of AI-assisted peer review, advocating for responsible adoption and ethical guidelines.
Artificial Intelligence in Peer Review: Enhancing Efficiency While Preserving Integrity
Review evaluating AI in peer review, noting benefits like efficiency but risks of over-reliance, bias, and ethical concerns; recommends AI as a supportive tool under human oversight.
AI-powered peer review process
Study with 80 CS students found genAI-powered code review increased engagement and identified more code issues in shorter times, leading to more fixes.