When two meta-analyses reach different conclusions, systematically compare their methodologies, inclusion criteria, and data sources to identify what drove the divergence.
Compare the Study Selection Criteria
Start by examining which studies each meta-analysis included or excluded. Different inclusion criteria—such as date ranges, study designs (RCTs only vs. observational studies), sample sizes, or population characteristics—can dramatically affect outcomes. Check if one meta-analysis applied stricter quality filters or included unpublished studies that the other missed.
Examine the Statistical Methods
Meta-analyses use different statistical models (fixed-effect vs. random-effects) and heterogeneity measures that can yield conflicting results from the same data. Look at how each handled study weighting, dealt with outliers, and assessed publication bias. If one shows high heterogeneity (I² > 75%), its pooled estimate may be less reliable.
Check the Recency and Coverage
A newer meta-analysis typically includes more recent studies and benefits from larger sample sizes. However, older analyses might have been more selective or focused on higher-quality evidence. Compare publication dates and the number of studies included—a 2024 meta-analysis with 45 studies will likely be more comprehensive than a 2019 analysis with 12 studies.
Assess Risk of Bias
Review each meta-analysis's quality assessment of individual studies. If one included multiple low-quality studies without sensitivity analysis, its conclusions may be less trustworthy. Look for GRADE ratings or similar quality indicators.
Look for Systematic Reviews
If the meta-analyses conflict significantly, search for umbrella reviews or systematic reviews of meta-analyses on your topic. These higher-order syntheses explicitly compare competing meta-analyses and can clarify which evidence is most robust. WisPaper's Scholar Search can help you quickly locate these umbrella reviews by understanding complex queries like "umbrella review OR overview of meta-analyses on [your topic]" while filtering out unrelated results.
Consider Context and Nuance
Sometimes both meta-analyses are "correct" but answer slightly different questions due to population differences, outcome measures, or intervention variations. The truth may be that effects vary by subgroup rather than one analysis being definitively wrong.

