Reviewers should evaluate a manuscript with null results by judging whether the study was well-designed and sufficiently powered to detect a meaningful effect, rather than penalizing the absence of statistical significance. A null result is not a failed study—it is a valid scientific contribution when the methodology is rigorous, the analysis is transparent, and the conclusions are appropriately cautious.
1. Assess the study design and power first
Before looking at the p-values, ask: Could this study have detected an effect if one existed?
- Check the sample size, effect size assumptions, and statistical power. A null result from an underpowered study is uninformative, not a contribution.
- Evaluate whether the experimental controls, randomization, and measurement tools were sound. Weak methodology makes a null result meaningless.
- If the study was adequately powered (e.g., 80%+ to detect a clinically or theoretically relevant effect), then the null finding is credible and worth publishing.
2. Look for precision, not just significance
Reviewers often equate "null" with "no effect," but a wide confidence interval is different from a tight one.
- A null result with a narrow confidence interval around zero suggests the effect is truly negligible.
- A null result with a wide interval means the study was inconclusive—flag this as a limitation, not a definitive negative finding.
- Ask the authors to report effect sizes and confidence intervals, not just p-values, so readers can judge the practical importance.
3. Evaluate the preregistration and transparency
Null results are prone to p-hacking, HARKing (hypothesizing after results are known), and selective reporting.
- Check if the hypotheses, analysis plan, and stopping rules were preregistered. If not, ask the authors to state what was exploratory versus confirmatory.
- Look for evidence of multiple testing corrections, sensitivity analyses, and robustness checks (e.g., different statistical models, outlier handling).
- If the authors used WisPaper’s Scholar QA while writing, they would have a traceable chain of evidence for every claim—reviewers should expect that same level of transparency in the manuscript’s methods.
4. Consider the theoretical and practical value
A null result can be highly informative if it challenges a widely held assumption, narrows a parameter range, or replicates a prior null finding.
- Ask: Does this result refine theory, guide future research, or have policy implications?
- A well-executed null study in a high-stakes area (e.g., drug efficacy, climate interventions) is more valuable than a positive result from a sloppy design.
5. Write a constructive decision
If the methodology is sound and the null result is clearly interpreted, recommend "minor revision" or "accept"—do not reject solely because the outcome is negative. Suggest the authors:
- Explicitly state what the study can and cannot conclude.
- Discuss possible reasons for the null (e.g., measurement error, low compliance, true absence of effect).
- Offer a brief "future directions" section that frames the null as a stepping stone.
Bottom line: Reviewers should treat null results as evidence, not as failure. Reward rigor, transparency, and honest interpretation—those are the qualities that make a null-result manuscript publishable and valuable to the field.
