An AI rarely replies, “That paper is imaginary.” More often it searches for something nearby, repairs the title without saying so, or produces a plausible-looking citation from familiar pieces.
I tested the failure mode with five deliberately invented paper titles. Each sounded reasonable and combined real research vocabulary, but none was copied from an existing record. I searched the exact titles in Crossref and OpenAlex, then searched distinctive title fragments.
None of the five had an exact record. The fragment searches still returned convincing neighbors. That is the dangerous part: a nearby real paper can make the invented citation feel confirmed when the title, authors, or study are different.
What does an AI do with an impossible request?
The response usually falls into one of four patterns:
- confident fabrication: it supplies title, authors, journal, year, and DOI-like text
- silent substitution: it returns a real paper on the topic without saying it changed the request
- partial match: it combines the title of one paper with the authors or journal of another
- useful refusal: it says no exact record was found and offers search terms or nearby literature
Only the fourth behavior keeps the evidence boundary clear. A citation is not verified because its pieces look academic.
How do you check a suspicious paper?
Start with the exact title in quotation marks. Then check the DOI resolver, publisher page, Crossref record, or a field database. Compare all of these fields together:
- title
- first author
- year
- journal or venue
- DOI or stable identifier
One matching field is not enough. If the DOI opens a different title, trust the DOI record and treat the original reference as broken.
When there is no exact match, search distinctive fragments and author combinations. This may reveal a transcription error or a real paper the answer distorted. Keep the corrected paper only after reading its abstract and confirming that it supports the intended claim.
Give the AI a retrieval rule before asking
Use a prompt such as:
Return only papers you can identify with a working DOI, publisher page, PubMed ID, or repository record. If you cannot verify an exact paper, say so. Do not complete missing citation fields from memory. Separate exact matches from related suggestions.
This does not guarantee accuracy. It makes uncertainty visible and gives you fields to check.
Where does WisPaper fit?
Search the title or research question in WisPaper Search. If you received a BibTeX file from an AI tool, upload it to TrueCite to flag references that do not validate against academic databases. Then open the underlying record before citing it.
The goal is not to rescue every citation. Sometimes the correct action is to delete it and find evidence again from the claim.
FAQs
Keep Reading
Can an AI Agent Find a Research Gap? Here’s How to Check Its Answer
Learn how to test an AI-generated research gap against search coverage, nearby terminology, recent papers, and counterexamples.
Can AI Build a Useful Literature Matrix? A 10-Paper Test
Build a useful AI literature matrix by defining decision-ready columns, preserving missing data, and verifying high-impact cells.
Can AI Read Tables in Research Papers Accurately?
Test AI table reading at the cell level and preserve headers, units, denominators, time points, confidence intervals, and footnotes.