You can check whether a paper's findings have been replicated by following its cited-by links to later papers, screening titles and abstracts for replication language, and recording the replication's outcome and sample before you rely on the original. Google Scholar documents cited-by links as a way to see work that cites a given paper, and Semantic Scholar's Academic Graph API provides citation data you can query for the same purpose.
Step-by-step check
- Open the original paper's cited-by list. In Google Scholar, use the cited-by link on the paper's record; through the Semantic Scholar API, retrieve the paper's citations. Expected result: a list of later papers that reference the original.
- Sort or filter toward later work. Google Scholar documents date sorting, which helps you move past early commentary to studies published after the original. Expected result: a shorter list weighted toward follow-up studies.
- Screen titles and abstracts for replication language. Look for terms such as "replication," "reproduce," "failed to replicate," or "direct replication." Expected result: a shortlist of candidate replication studies, with general citations set aside.
- Read the replication's outcome and sample. Note whether the replication reports a consistent or inconsistent result and what sample it used. Expected result: a decision on whether the replication bears on the original finding for your context.
- Widen the search if the cited-by list is thin. The Cochrane Handbook describes searching reference lists and other sources and documenting the search. Expected result: a documented trail you can describe in your manuscript or review.
What changes your decision
A replication with a different sample or population may not test the same claim as the original, so match the replication's sample to the population your manuscript addresses. A citation that mentions the original without reporting a replication attempt does not establish replication; treat it as unresolved and check the citing paper's methods. If no replication appears in the cited-by list, the status remains unresolved rather than disproven, and the next step is to search the topic directly and check reference lists of relevant reviews.
