Why a single accuracy score can mislead you completely
A single metric—like asking one AI model to judge another—can produce rankings that are the exact opposite of reality. In a 2026 study of six medical AI research systems, a single-model evaluation ranked the best-performing system last (score 55.5), while a three-tier framework that combined programmatic citation verification, rule-based compliance checks, and multi-model judging ranked it first (score 81.8) [2]. This complete reversal shows that subjective, single-judge evaluations are unreliable for measuring factual accuracy.
The same study found that citation integrity was the decisive quality dimension: hallucination rates ranged from 2.9% to 36.8% across systems, and a hard-rule threshold on per-task citation scores capped four of six systems' total scores at the penalty ceiling [2]. Adding a multi-agent citation verification and repair pipeline improved one system's citation integrity score from 40.0 to 90.9, raising its weighted total from 68.9 to 81.8 [2]. This means that automated verification pipelines can dramatically improve accuracy, but only if they are built into the evaluation framework from the start.
The gap between best-case and typical accuracy is wide
Even the best AI models in these studies produce a citation error roughly one out of every four times. In a 2026 pilot evaluation of four AI models generating PubMed-style references for eye disease research, DeepSeek achieved the highest accuracy at 78.6% (22 out of 35 citations correct), followed by ChatGPT and Copilot at 51.4% each, and Gemini at just 12.9% [1]. A separate 2025 study on neuro-ophthalmology citations found similar results: DeepSeek at 75.0%, Copilot at 60.5%, ChatGPT at 31.4%, and Gemini at 3.0% [5]. The most common errors were DOI mismatches and the generation of irrelevant or unverifiable references—what researchers call 'hallucinations.'
These numbers come from controlled tasks with standardized paragraphs, so real-world performance may be worse. The 2026 study noted that expert validation confirmed DeepSeek's relative advantage, with 42.9% of its references classified as fully cited, compared to 20.0% for Copilot and 11.4% for ChatGPT and Gemini [1]. This means that even the best model only produced fully correct citations less than half the time when judged by human experts. Across all five studies, the consistent finding is that no AI system achieves the 95%+ accuracy needed for scholarly work without human verification.
Human oversight is not optional—it is the only safety net
All five studies converge on the same conclusion: AI citation tools require rigorous human verification. A 2026 study of orthopaedic surgery literature found that 27.5% of 229 references contained errors, with 9.2% being major errors like contradictory conclusions or missing facts [3]. When the researchers tested two AI platforms to automate citation accuracy checks, both failed to complete a thorough review because they could not access full-text sources or apply the classification system [3]. This means that AI tools cannot even reliably check their own work, let alone fix errors in existing literature.
Student evaluations of ChatGPT and Jenni AI in a 2026 study showed that while students recognized AI's potential to enhance writing efficiency, they emphasized the need for human verification to ensure factual correctness and ethical compliance [4]. Jenni AI demonstrated greater consistency and citation verification than ChatGPT, which exhibited more frequent fabrication and inaccuracy [4]. The practical takeaway is that any deployment of citation-checking AI must include a human-in-the-loop who can access full-text sources, apply a structured error classification system, and make final judgments on accuracy.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2025 to 2026, 5 from 2024 or later — selected as the most relevant from 5 studies that passed quality screening, drawn from 55 papers retrieved from a database of over 500 million.
Sources used in this answer
Evaluation of AI Citation Accuracy in Anterior Segment Research.
In a pilot evaluation of four AI models generating PubMed-style references for eye disease research, DeepSeek had the highest accuracy (78.6%), followed by ChatGPT and Copilot (51.4% each), and Gemini (12.9%); expert validation confirmed DeepSeek's advantage with 42.9% fully cited references, but all models showed substantial error rates including hallucinations.
Citation Hallucination Determines Success: An Empirical Comparison of Six Medical AI Research Systems
In a benchmark of six medical AI research systems using programmatic citation verification, hallucination rates ranged from 2.9% to 36.8%; a single-model evaluation ranked the best system last (55.5) while a three-tier framework ranked it first (81.8), showing that multi-layered evaluation is essential for accurate assessment.
Citation Inaccuracies in Orthopaedic Surgery: A Novel Classification and Precautions for AI-Generated Bibliographies
A review of 229 references in orthopaedic surgery literature found 27.5% contained errors (9.2% major errors); two AI platforms failed to complete a thorough review because they could not access full-text sources or apply the proposed citation accuracy classification system.
AI Tools for Academic Integrity: A Student Response to Chat GPT and Jenni AI in Citation Accuracy
In a student evaluation of ChatGPT and Jenni AI for citation generation, students recognized AI's efficiency but stressed the need for human verification; Jenni AI showed greater consistency and fewer fabrications than ChatGPT.
Benchmarking Artificial Intelligence Models for Citation Accuracy in Neuro-Ophthalmological Disorders Research: A Comparative Analysis of Four Models
In a comparative analysis of four AI models for neuro-ophthalmology citations, DeepSeek achieved 75.0% accuracy, Copilot 60.5%, ChatGPT 31.4%, and Gemini 3.0%; expert review found DeepSeek produced 15 fully cited references versus 7 for Copilot and 4 each for ChatGPT and Gemini.