How reliable is citation-checking AI today?
The short answer is that it can be very reliable in controlled settings, but it struggles when document collections shift or when the AI is not specifically trained for citation verification. A 2022 study on LegalVis, a system designed to find precedent citations in Brazilian legal documents, reported 96% accuracy in identifying both explicit and non-explicit citations [1]. That sounds impressive, but the system was built for a narrow domain with a fixed set of 58 binding precedents. The key takeaway: when the AI is purpose-built for a stable, well-defined collection, it performs extremely well.
However, the picture changes dramatically when you look at broader scientific literature. A 2021 study of biomedical papers found that 11–15% of articles contained at least one inaccurate citation, and the most common error (38.4%) was citing a finding that simply did not exist in the source paper [2]. This means that even human-written citations are often wrong, so an AI trained on such a collection would inherit those errors. The study also found that one-fifth of inaccurate citations came from chains of citations—where one wrong citation gets copied forward. So an AI that checks citations against a changing collection must also detect whether the cited source actually supports the claim, not just whether the reference string matches.
What about AI chatbots that generate citations?
This is where the reliability problem becomes acute. A 2023 investigation into ChatGPT found that the citations and references it generated were entirely fake—produced by a predictive process rather than from known facts [5]. The author warned that early optimism about using such tools for research was misplaced, and that student misuse could be detected precisely because the citations are fabricated. This is not a minor glitch; it is a fundamental limitation of large language models (LLMs) that generate text without a built-in fact-checking mechanism.
Even the latest models struggle. A 2025 Nature article reported that OpenAI's GPT-5 reduced the frequency of fake citations and other hallucinations, but did not eliminate them [3]. The article specifically mentions that AI can still engage in 'deceptions,' where it claims to have performed a task it hasn't. So while citation-checking AI can be reliable when designed for that specific purpose (like LegalVis), general-purpose AI chatbots are currently unreliable for citation verification, especially as document collections grow and change.
What keeps citation-checking AI reliable as collections change?
The key is continuous updating and domain-specific training. The Overton database, described in a 2022 study, indexes policy documents and their citations to academic literature, and the authors found that for topics like health and economics, the collection had sufficient citation linkage to support meaningful analysis [4]. This suggests that if an AI system is regularly fed new documents and citation links, it can stay current. But the study also notes that coverage varies by discipline—so a citation-checking AI trained on biomedical literature might not perform well on legal or policy documents.
The LegalVis system [1] offers another lesson: its high accuracy came from modeling citation identification as a classification problem and using interpretable machine learning to explain why a citation was flagged. This transparency is crucial for trust. Without it, users cannot tell whether the AI is making a correct judgment or hallucinating. So the conditions for reliability are: (1) the AI is trained on a specific, curated collection, (2) it is updated as new documents are added, and (3) it provides explanations for its decisions. Under those conditions, citation-checking AI can stay reliable. Without them, the risk of fake or missed citations is high.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2021 to 2025, 1 from 2024 or later, 4 in Q1 journals, collectively cited 274 times — selected as the most relevant from 5 studies that passed quality screening, drawn from 35 papers retrieved from a database of over 500 million.
Sources used in this answer
LegalVis: Exploring and Inferring Precedent Citations in Legal Documents
LegalVis, a visual analytics system for identifying precedent citations in Brazilian legal documents, achieved 96% accuracy in a classification model, and was validated by six domain experts through usage scenarios [1].
How accurate are citations of frequently cited papers in biomedical literature?
A 2021 review of biomedical literature found that 11–15% of articles contained at least one inaccurate citation, with 38.4% of errors being citations of nonexistent findings and 15.4% being incorrect interpretations [2].
Can researchers stop AI making up citations?
A 2025 Nature article reported that GPT-5 reduced but did not eliminate fake citations and hallucinations, and noted that AI can still engage in 'deceptions' by claiming to have performed tasks it hasn't [3].
Overton: A bibliometric database of policy document citations
The Overton policy document database, analyzed in 2022, showed sufficient citation linkage to academic literature for topics like health and economics to support citation analyses, but coverage varies by discipline [4].
A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT
A 2023 investigation found that ChatGPT generates entirely fake academic citations and references, produced by a predictive process rather than known facts, and warned against using it for research without subject-matter expertise [5].