When two studies on the same question point in different directions, the temptation is to average them, pick the one that matches your prior, or write a vague "the evidence is mixed." None of those helps a clinical or research audience. Writing well about disagreement means showing where the studies diverge, why that divergence matters for a defined population and outcome, and what a reader should do next.
This guide gives you a complete sequence for producing a focused evidence brief on a disagreement. It uses one running example throughout — a hypothetical question about whether a specific intervention improves a defined outcome in a defined population — and it separates four things that are often blurred together: evidence level, applicability, indirectness, and limitations. It ends with a reusable checklist and explicit completion criteria so you know when the brief is ready to circulate.
What does a reader actually need from a disagreement brief?
A reader preparing a research update or team discussion is not asking "which study is right?" They are asking a narrower set of questions:
- What exactly is the question, and in whom?
- Which studies address it, and how do they differ in design and conduct?
- Where do the results actually conflict — and where do they only appear to?
- How directly does each study speak to this population and this outcome?
- What are the limitations that could explain the conflict?
- What is the defensible bottom line, stated with appropriate scope?
Your brief succeeds when a reader can answer those questions without opening every paper. It fails when it reports a vote count ("three for, two against") or implies a consensus the evidence does not support.
Step 1: Fix the question before you read anything
Disagreement is often an artifact of a fuzzy question. Before searching, write a one-sentence question that names the population, the intervention or exposure, the comparator, and the outcome. Keep it stable across the whole brief.
For the running example, the question might be: In adults with a defined chronic condition, does the intervention improve the specified clinical outcome compared with usual care?
This step is not bureaucratic. Cochrane guidance on searching recommends planning searches directly from eligibility criteria, so the question and the search are built from the same definition. If your question drifts mid-brief, your inclusion decisions drift with it, and apparent disagreement can come from comparing studies that answer different questions.
Checkpoint: You can state the question in one sentence, and a colleague could use it to decide whether a given paper belongs in the brief.
Step 2: Search broadly and document the process
A brief on disagreement is only as good as the set of studies it considers. Cochrane guidance recommends using multiple sources, combining free-text terms with subject headings, documenting the process, deduplicating records, and screening in stages. It also recommends linking multiple reports of the same study so a single trial reported in several papers is not counted as several independent findings — a common source of phantom disagreement.
PRISMA 2020 provides reporting structures for systematic reviews, including transparent reporting of search, selection, and synthesis methods, plus flow-diagram templates. Even a lightweight brief benefits from borrowing that discipline: record your sources, your terms, your screening decisions, and why records were excluded.
In practice, you can run structured scholarly searches through WisPaper search and keep the query terms aligned with your eligibility criteria. If you want a public entry point to explain the search approach to collaborators, the WisPaper Scholar Search page describes finding academic papers.
Checkpoint: You have a documented list of sources and terms, a deduplicated record set, and a note on which reports belong to the same underlying study.
Step 3: Organize the evidence so disagreement becomes visible
Once records are screened, the work shifts from collecting to structuring. Save the included papers in one place so the same set feeds your table, your synthesis, and any follow-up questions. My Library is where you can save or upload papers and organize them for this purpose.
For a larger brief with multiple sub-questions or a team, Research Projects gives you a project area to hold the work. The point is not the tool; it is that every claim in your draft traces back to a paper in a defined set.
Checkpoint: Every study you plan to discuss is in one organized set, and you can name the set's boundaries (what is in, what is out, and why).
Step 4: Separate evidence level, applicability, indirectness, and limitations
This is the step where most disagreement briefs go wrong. These four concepts answer different questions, and collapsing them produces muddled prose.
| Concept | Question it answers | What to record per study |
|---|---|---|
| Evidence level | How strong is the design for causal questions? | Study design and its position in your chosen hierarchy |
| Applicability | Does the studied setting resemble the setting where the reader will use this? | Population, care setting, comparator, and outcome definitions |
| Indirectness | How far is the studied comparison or outcome from the question you asked? | Differences in intervention, comparator, population, or outcome |
| Limitations | What could bias this study's result? | Conduct issues, missing data, and other weaknesses you can document |
A high-level study can still be indirect for your question; a highly applicable study can have serious limitations. When you write, name which of these is driving the disagreement. "The trials disagree" is not an explanation. "The two trials used different outcome definitions, and one enrolled a broader population" is.
Checkpoint: For each study in your table, you have filled all four columns — and you can point to the specific feature, not a general impression.
Step 5: Map the disagreement before you explain it
Before drafting prose, build a disagreement map. For each pair or cluster of conflicting findings, record:
- The exact result that conflicts — the outcome, the direction, and the comparison.
- Whether the conflict is real or apparent — different outcomes, different comparators, or duplicate reports can create the appearance of conflict.
- The most plausible driver — design, applicability, indirectness, or limitations.
- What would resolve it — a study feature, a subgroup, or a synthesis method.
This map becomes the skeleton of your brief. It also protects you from over-claiming: if you cannot name a driver, say the disagreement is unexplained rather than inventing a cause.
Checkpoint: Every conflict in the brief has a row in the map, and every row has either a named driver or an explicit "unexplained."
Step 6: Draft the brief so each section does one job
A workable structure for a focused brief:
- Question and scope — the one-sentence question, the population, and the outcome.
- Evidence set — how many studies, of what designs, and how they were found.
- Where the studies agree — state this first; it frames the disagreement.
- Where they disagree — one subsection per conflict, each tied to a row of your map.
- Why it matters for this population — applicability and indirectness, stated concretely.
- Limitations of the evidence base — study-level and set-level.
- Bottom line — what a reader can reasonably conclude, scoped to the question you asked.
Keep the bottom line narrower than the evidence. If the studies disagree, the honest bottom line often describes the disagreement and what would resolve it, rather than declaring a winner.
Checkpoint: Each section maps to one job, and no section mixes evidence level with applicability or limitations.
Worked example: writing the disagreement paragraph
Suppose your map has two rows. In row one, a larger trial reports benefit on the specified outcome, while a smaller trial reports no clear difference. In row two, both trials used the same comparator but defined the outcome differently.
A weak draft says: "The evidence is mixed; one trial was positive and one was not."
A stronger draft does four things. It states the question and population. It reports the direction of each result against the same outcome framing. It names the outcome-definition difference as a driver of the apparent conflict. And it closes with what would resolve the question — for example, studies using a harmonized outcome definition in the same population.
Notice what the stronger draft does not do: it does not rank the trials by prestige, it does not average their results, and it does not tell an individual clinician what to do for a specific patient. It describes the evidence base and its boundaries.
Checkpoint: Read your paragraph aloud. If a listener cannot say where the studies diverge and why, revise.
Step 7: Use synthesis tools at the point of need
Some steps in this workflow benefit from tooling; others do not. Match the tool to the task.
- Interrogating your own set: When you need to ask questions grounded in the papers you have already collected, Library QA lets you ask questions based on papers uploaded to My Library or added from search results. Use it to check whether a claim in your draft is actually supported by a paper in your set.
- Longer synthesis tasks: For a broader literature review or a deeper synthesis that goes beyond a single brief, Scholar Agent fits literature review and related research tasks.
- Reference hygiene: Before circulating a brief with a bibliography, screen your BibTeX with TrueCite, which checks BibTeX against real academic databases to flag hallucinated references.
Each of these supports a specific step. None of them replaces the judgment calls in Steps 4 and 5 — deciding what counts as a driver of disagreement is your analytical work.
Checkpoint: Every tool you used maps to a step in the workflow, and you can say what it contributed.
Step 8: Apply the reusable checklist before you circulate
Use this checklist as a final pass. Each item should be answerable with a yes and a pointer to where in the brief it is satisfied.
- The question names population, intervention or exposure, comparator, and outcome.
- The search sources and terms are documented.
- Multiple reports of the same study are linked, not double-counted.
- Each study has entries for evidence level, applicability, indirectness, and limitations.
- Every conflict has a row in the disagreement map.
- Each conflict is labeled real or apparent, with a reason.
- Each conflict has a named driver or an explicit "unexplained."
- The bottom line is scoped to the question and does not exceed the evidence.
- The brief avoids individual treatment recommendations.
- References have been screened for accuracy.
Checkpoint: Every box is checked, and a colleague who did not write the brief could verify each one from the document itself.
How do you know the brief is ready?
A disagreement brief is ready when all of the following are true:
- The question is stable. It did not change between search and draft.
- The evidence set is bounded. You can state what is in and what is out, and why.
- The disagreement is mapped. Every conflict has a location, a status (real or apparent), and a driver or an explicit gap.
- The four concepts are separated. Evidence level, applicability, indirectness, and limitations each appear where they belong.
- The bottom line is honest. It describes what the evidence supports for the stated population and outcome, and it names what would resolve remaining uncertainty.
- The references are clean. Citation screening is complete.
If any of these is missing, the brief is a draft, not a finished product. Circulating a draft that implies consensus is worse than circulating nothing.




