Why do AI detectors falsely accuse legitimate writers?
The most damaging evidence gap is the high rate of false positives — cases where a detector labels human-written text as AI-generated. A 2024 study of scholars from multiple institutions found that false positives disproportionately affect non-native English speakers and writers with distinctive styles, leading to unwarranted accusations that can harm academic careers [5]. This isn't a minor edge case: the same study identified algorithmic biases, vulnerability to manipulation, and a lack of contextual understanding as built-in flaws of current tools [5]. A separate 2025 analysis of three popular detectors (GPTZero, ZeroGPT, Corrector App) on 1,000 texts found that while they achieved moderate to high accuracy (AUC scores of 0.75 to 1.00), none reached 100% reliability, meaning false positives are an inherent risk [1]. Together, these studies show that the evidence base for how often false positives occur across different populations is still thin, and the tools are not safe to use as sole arbiters of academic integrity.
Can detectors actually catch AI-written content?
A second critical gap is that traditional plagiarism detectors — which rely on matching exact strings of text — are nearly blind to AI-generated content. A 2023 study created 80 academic writing samples using GPT-based tools and found that conventional similarity-based anti-plagiarism software failed to detect them because machine-generated text has different syntax and structure than human writing [7]. AI-specific detectors can catch some of this content by looking for telltale patterns, but they are far from perfect [7]. In code plagiarism, even the best large language model tested (GPT-4o) achieved only 78.70% accuracy and an 86.97% F1 score, with higher false positive rates flagged as an inherent limitation [2]. A 2024 study on modeling assignments showed that AI-obfuscated plagiarism (where students use ChatGPT to disguise copied work) required a combined automated-and-human approach to achieve significantly better detection than state-of-the-art tools alone [8]. The evidence gap here is clear: no single detector or approach has proven reliable enough to stand alone against AI-generated or AI-obfuscated content.
What about plagiarism in languages other than English?
A third major gap is the near-total lack of evidence for detectors working in low-resource or mixed-language settings. A 2025 study on bilingual Sinhala-English plagiarism found that popular tools like Turnitin simply do not handle Sinhala text, and cloud-based services raise data privacy concerns for local universities [3]. The researchers built a custom system using TF-IDF and cosine similarity that achieved over 85% accuracy on student essays, but this is a prototype, not a widely tested solution [3]. A broader 2025 review of 189 papers on plagiarism detection concluded that the field has made progress in identifying highly obscured plagiarism, but the most promising future direction is combining multiple analytical methods — textual and non-textual — and that work on non-English and bilingual contexts remains sparse [6]. This means that for the millions of students and researchers working in languages other than English, the evidence base for effective detection is essentially absent.
About These Sources
This answer is built on 8 peer-reviewed studies — published from 2023 to 2025, 7 from 2024 or later, 1 in Q1–Q2 journals, collectively cited 60 times — selected as the most relevant from 11 studies that passed quality screening, drawn from 63 papers retrieved from a database of over 500 million.
Sources used in this answer
Can we trust academic AI detective? Accuracy and limitations of AI-output detectors
In a study of 1,000 texts (250 human, 750 ChatGPT-generated), three AI detectors (GPTZero, ZeroGPT, Corrector App) achieved AUC scores of 0.75–1.00 but none reached 100% reliability, confirming false positives are a persistent risk [1].
Can Large Language Model Detect Plagiarism in Source Code?
GPT-4o achieved the highest accuracy (78.70%) and F1 score (86.97%) for detecting code plagiarism, but open-source models like LLaMA 3 matched it on complex cases; higher false positive rates were noted as an inherent limitation [2].
A Bilingual Plagiarism Detection Approach for Sinhala–English Text Using TF-IDF and Cosine Similarity
A bilingual Sinhala-English plagiarism detection system using TF-IDF and cosine similarity achieved over 85% accuracy on student essays, but remains a prototype with limited generalizability [4].
AI on Academic Integrity and Plagiarism Detection
AI-powered tools significantly improve detection of paraphrased and AI-generated text, but raise concerns about over-reliance on automated assessments and the need for human oversight and policy frameworks [5].
The Problem with False Positives: AI Detection Unfairly Accuses Scholars of AI Plagiarism
False positives from AI detection tools disproportionately affect non-native English speakers and scholars with distinctive writing styles, leading to unwarranted accusations and a climate of distrust [6].
Comparative analysis of text-based plagiarism detection techniques.
A systematic review of 189 papers (2019–2024) found that combining multiple analytical methods (textual and non-textual) is the most promising future direction for plagiarism detection, but non-English and bilingual contexts remain underexplored [7].
Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?
Traditional similarity-based anti-plagiarism tools failed to detect 80 GPT-generated academic writing samples due to differences in syntax and structure, while AI detector tools showed some success based on machine-specific characteristics [9].
Automated Detection of AI-Obfuscated Plagiarism in Modeling Assignments
A combined automated-and-human inspection approach achieved significantly higher detection rates for AI-obfuscated plagiarism in modeling assignments than state-of-the-art tools alone [11].