Publication bias occurs when studies with positive or statistically significant results are more likely to be published than those with null or negative findings, and you can account for it in your literature review by systematically searching for unpublished work, assessing the risk of bias in your included studies, and using statistical methods like funnel plots and trim-and-fill analysis to detect and adjust for its effects.
Start with a systematic, transparent search strategy
The first line of defense against publication bias is ensuring your literature review isn’t limited to what’s already published in major journals. Search multiple databases (e.g., PubMed, Scopus, Web of Science), but also include:
- Preprint servers (arXiv, bioRxiv, SSRN) where studies appear before journal acceptance.
- Clinical trial registries (ClinicalTrials.gov) to find completed but unpublished trials.
- Conference proceedings and dissertations via ProQuest or OpenGrey.
- Citation chasing — check reference lists of included studies for older or obscure work.
Document every source you searched and your search terms. This transparency allows readers to judge how thoroughly you attempted to capture the grey literature.
Assess and report risk of bias in included studies
Publication bias is one form of bias, but it interacts with others. Use a tool like ROBINS-I (for non-randomized studies) or the Cochrane Risk of Bias tool to evaluate each included study. Studies with small sample sizes, weak designs, or industry funding are more prone to selective reporting. Report these assessments in a table so readers can see the overall quality of your evidence base.
Use statistical methods to detect and correct for bias
If your review includes quantitative data from at least 10–15 studies, you can formally test for publication bias:
- Funnel plot — a scatterplot of effect size versus study precision. Asymmetry suggests missing studies, often on the null side.
- Egger’s regression test — a statistical test for funnel plot asymmetry.
- Trim-and-fill method — estimates how many studies are missing and adjusts the pooled effect size accordingly.
- Fail-safe N — calculates how many unpublished null studies would be needed to overturn your conclusion.
Report the results of these tests even if they show no bias — this strengthens your review’s credibility.
Discuss the direction and magnitude of the bias
Even if you can’t fully eliminate publication bias, you can describe its likely impact. For example, if your funnel plot is asymmetric and trim-and-fill adjusts your effect size downward, state this explicitly. Acknowledge that your conclusions may overestimate the true effect.
Practical tip
If you’re managing a large number of references and need to track which studies came from which search source, WisPaper’s My Library can help you organize grey literature and preprint files alongside published papers, with automatic metadata extraction to keep your bias-assessment table accurate. This makes your audit trail easier to maintain and report.
