A p value answers one narrow question: how surprising is this result if the null hypothesis were true? An effect size answers a different and often more useful question: how large or meaningful is the observed difference or relationship? When a research or analysis team prepares a client deck, decision memo, or study report, those two questions get conflated almost immediately. One stakeholder reads "p < 0.05" as "the effect is real and worth acting on." Another reads the same line and asks, "Yes, but how much did it actually move?"
This article assumes you are preparing a traceable research deliverable, and it gives you a practical way to report both statistics so a reviewer can see the size of the finding and the strength of the evidence in the same table row. It covers the common reporting failure modes, a decision workflow for choosing what to present, a before-and-after example built around a decision memo, and a concrete QA checklist you can hand to a colleague before submission.
What does the reader actually need each number to do?
A p value supports a yes/no style decision about a specific hypothesis under specific assumptions. It tells the reader whether the observed pattern is unlikely under a null model. It does not tell the reader whether the pattern matters for the stated research question, how precisely the effect was estimated, or whether the finding would replicate in a new sample.
An effect size supports a magnitude question. It lets the reader compare findings across studies, judge practical relevance, and decide whether a statistically detectable difference is worth acting on. In a client report, the effect size is often the number the decision-maker needs; the p value is the number the methodologist needs in order to trust that the effect size was not produced by noise.
The cleanest way to keep them from competing is to treat them as completing different parts of one sentence. The p value answers "should I take this estimate seriously, given the sample and the analysis?" The effect size answers "if I take it seriously, what is the actual size of what I am looking at?" That sentence-level framing is what most reporting checklists are trying to enforce when they ask for both.
Why do reporting standards insist on both?
Discipline-level reporting standards do not treat effect size and p value as substitutes. The APA Journal Article Reporting Standards ask authors to provide enough detail for readers to evaluate how a study was conducted and analyzed, and the design-specific modules press for reporting practices that make the magnitude and precision of results visible. A report that offers only a p value leaves a reader unable to evaluate how large the finding was; a report that offers only a raw difference leaves a reader unable to evaluate whether that difference could be explained by sampling variation.
The same logic appears in reproducibility guidance. NIH frames scientific rigor as the strict application of the scientific method, including transparent design, methods, analysis, interpretation, and reporting. An analysis that is transparent about design but silent about effect magnitude is not fully reported, because the reader cannot connect the statistical decision to the practical question the study was meant to answer.
For observational work, STROBE checklists ask authors to report results in a form that lets readers assess both the estimate and its uncertainty. The checklist structure pushes a researcher to state the outcome, the exposure, the estimate, and the measure of precision together, rather than letting significance testing stand alone as the summary of the finding.
What goes wrong when reports only feature the p value?
The most common failure mode is the p value that becomes the headline. When a team writes "the treatment effect was statistically significant (p = 0.03)," the reviewer still cannot tell whether the observed improvement was 1 percent or 40 percent. In a large enough sample, nearly any tiny difference can clear a conventional threshold. The p value is doing its job; the report is not.
The reverse failure mode is also common: a report presents an impressive percentage change with no precision information. A stakeholder sees "reduced cost by 18 percent" and cannot tell whether that estimate is based on a stable, repeated pattern or on a handful of unrepresentative observations. Both failure modes come from the same error: treating one statistic as a summary of the finding rather than as one part of the finding.
A third failure mode is mismatched interpretation. A team may compute a small effect size and then write interpretive language that sounds large, or compute a large effect size and then bury it behind cautious null-hypothesis phrasing. The reviewer's job is to compare the numbers you report against the claim you draw. When those do not line up, the entire artifact loses credibility even if every number is correct.
Which effect size goes with which research question?
Choosing an effect size is a research-decision, not a formatting decision. For a comparison between two group means, a standardized difference such as Cohen's d lets the reader see the gap in pooled standard deviation units. For a correlation-style question, the correlation coefficient itself is an effect size. For a difference in proportions, the risk difference or an odds ratio may be more interpretable depending on the audience. For a model-based analysis, a regression coefficient in the original units is often the most decision-ready number, because it connects directly to the metric the stakeholder tracks.
The decision depends on what the reader needs to do with the number. If the stakeholder asks "how much did the new onboarding flow reduce time to first value?", then a coefficient in hours is more useful than a standardized d, even if the standardized version is easier to compare across studies. If the stakeholder asks "is this effect comparable to what we saw in the earlier pilot?", then a standardized measure helps. State that choice explicitly in the methods note so the reviewer can see why the reported effect size matches the decision.
What does a submission-ready results block look like?
A submission-ready table row usually contains the effect size estimate, an interval around it, the p value, and enough sample or model context to interpret both. In a decision memo, a single sentence above the table can explain the reporting choice: "We report the unstandardized difference in days because the operational decision requires a time value, and we report the 95% interval because the business owner needs to know the plausible range."
What makes the row reviewable is that the effect size and p value point in the same direction without being used interchangeably. A reviewer should be able to see a large effect with a small p value and understand that the finding is both practically relevant and unlikely to be noise. A reviewer should also be able to see a small effect with a small p value and understand that the analysis was powered to detect a modest, possibly unimportant difference. The table creates that distinction without requiring the reader to guess.
Decision memo example: onboarding change with provenance attached
Imagine an analysis team studying whether a revised onboarding flow reduces time-to-first-value for a software product. The team compares the revised flow against the previous flow across two groups. They find an unstandardized difference of 2.4 days in the revised group's favor, a 95% interval from roughly 1.1 to 3.7 days, and a p value of 0.002. The report states the confidence level for the interval explicitly.
The provenance-conscious version of this memo carries a source note under the table: the data source for the time measurement, the date range of the comparison, the analysis code version, and the reporting standard used to structure the table. The confidence note says something like: "We are confident in the direction of this difference; the operational question is whether a 2.4-day improvement justifies the implementation cost, and that is a business decision rather than a statistical one."
This example shows why the two statistics belong together. The p value tells the reviewer the difference is unlikely to be a sampling artifact. The effect size tells the decision-maker that the realistic improvement is measured in days, not in an abstract significance flag. A report that contained only one of those numbers would force the stakeholder to ask a follow-up question that the table should already have answered.
How do you check the result before you circulate it?
A pre-submission check should force you to read the effect size and p value in the same pass. For each headline finding, ask three questions. First, can the reader restate the effect size in the units of the original question? Second, can the reader see the precision around the estimate? Third, does the p value support the inferential claim being made about that estimate?
When the work is part of a larger evidence base, provenance matters as much as the numbers. Your QA step should confirm that every summary statistic traces to a specific source, that the source can be found again, and that no reference exists only as a stale or fabricated citation. A team maintaining a shared library can run that check systematically rather than relying on memory.
When does the combined workflow save a report?
There is a recurring situation where the combined workflow prevents a serious mistake: the stakeholder wants to oversell a significant p value from a tiny effect. The report writer has the burden of showing the effect size and letting the practical relevance question land where it belongs. A table that reports both numbers makes that conversation much shorter, because the mismatch between statistical significance and practical importance is visible in the same row.
There is a second recurring situation where the p value saves the effect size from being dismissed. A stakeholder sees a big percentage change and wants to act immediately. The precision information shows that the interval is wide and the finding is fragile. The effect size alone would have looked decisive. The p value, reported honestly, shows it is not.
The combined workflow is less about adding columns and more about controlling the interpretive frame. When both numbers are present, the report author decides what the reader compares. When one number is missing, the reader's prior belief fills the gap.
Decision table: when to lead with which number
The choice of what to feature depends on the decision in front of the reader. The table below summarizes the tradeoffs.
| Reporting situation | Lead with | Why | Common failure |
|---|---|---|---|
| Stakeholder needs an operational estimate, such as hours saved or dollars moved | Effect size in original units | The decision requires magnitude | Leading with significance hides the size |
| Reader must judge a methodological claim against prior literature | Effect size plus interval | Standardized or comparable units support cross-study judgment | Raw p value blocks comparison |
| Narrow confirmatory test of a prespecified hypothesis | P value with explicit threshold and context | The inferential claim is a yes/no question | Reporting the threshold without the estimate |
| Small sample or wide interval | Interval and precision prominently | The reader must see fragility | A single p value suggests false certainty |
| Large sample where tiny effects clear the threshold | Effect size first, p value as context | Significance alone misleads | Calling a trivial effect "important" |
No row says "report only one." The column is about which number anchors the prose and which number appears as supporting context.
Where does provenance fit in a reporting workflow?
A report is only as reviewable as its sources. When a team builds a deliverable from many papers, keeping the source list connected to the claims prevents the most damaging late-stage problem: a stakeholder challenges a number and nobody can reconstruct where it came from. A shared library where papers are organized by project makes that reconstruction quick. When a reviewer asks "which version of this guideline are we citing?", the answer should be visible in the project, not buried in a long-dead chat thread.
For a larger research effort, My Library lets the team save and upload papers into a single organized space, while Research Projects keeps a given deliverable's sources together as the project moves toward submission. Questions that come up mid-revision can be worked against the team's saved set with Library QA, which is built for asking questions based on papers already uploaded to My Library or added from search.
What does the final submission check look like?
Before the memo or deck leaves the team, run the following checks in order. First, confirm every headline finding reports an effect estimate and its precision together, and that no p value appears as a standalone summary. Second, confirm each effect size is labeled with its unit or its standardized form, so a reviewer can interpret the number without asking. Third, confirm the prose draws only the conclusions the estimates support; a wide interval must not be described as a precise finding.
Fourth, confirm the provenance chain. Every claim should point to a source the team can reopen, and every reference should be real and traceable. For teams working with BibTeX references, TrueCite checks entries against academic databases to flag hallucinated citations before the deliverable goes out. Fifth, confirm the reporting standard named in the methods matches the structure of the results section. A memo that says it follows a checklist but does not visibly follow its logic will draw a reviewer's immediate attention to the gap.
Finally, ask a colleague who was not on the analysis to read only the table and the conclusions, then restate the finding aloud. If they cannot state both the size and the strength of the evidence in one sentence, the table has not done its job.




