How to Build an Evidence Hierarchy for a Focused Clinical Question

A focused clinical question is not answered by finding the highest-ranked study type and stopping there. The practical task is to define the question precisely, retrieve the relevant evidence types, rank them by design and risk of bias, and then judge whether each source actually applies to your population and outcome. This article gives a step-by-step sequence for clinicians and medical researchers who need a defensible evidence hierarchy for research, guideline, or scholarly work — including diagnostic checkpoints, a worked example, a reusable checklist, and clear stop rules.

An evidence hierarchy is only meaningful relative to a question. Before searching, write down the population, the intervention or exposure, the comparator, and the outcome you care about. A hierarchy built for a broad topic will mix evidence that answers different questions.

Use a focused example throughout this article: in adults with community-acquired pneumonia managed as outpatients, does a short antibiotic course compared with a standard course change clinical cure? Fixing population and outcome this way lets you decide which study designs are even eligible, and it prevents a guideline recommendation for hospitalized patients from being counted as direct evidence for outpatients.

Cochrane guidance supports planning searches from explicit eligibility criteria, combining free-text terms with subject headings, using multiple sources, and documenting the process. That planning step is where the hierarchy begins: eligibility criteria determine which evidence levels can enter your ranking at all.

A hierarchy is only as good as the records you screen. Use field tags, phrase searching, filters, and Advanced Search in PubMed to structure the query, and inspect Search Details to see how terms were translated and combined. This makes the strategy auditable and repeatable.

Controlled vocabulary strengthens the strategy. MeSH is NLM's controlled and hierarchical vocabulary; preferred headings, entry terms, scope notes, qualifiers, and tree positions can be used to expand or focus a PubMed strategy. For the pneumonia example, MeSH tree positions help you decide whether to broaden to related respiratory infections or narrow to a specific population.

Cochrane also recommends deduplicating records, screening in stages, linking multiple reports of one study, and rerunning searches before publication. Each of these protects the hierarchy from double-counting the same trial under different reports.

If you want a scholarly search interface to run and refine this step, WisPaper search supports finding academic papers; the public Scholar Search page explains the entry point for finding papers.

Before ranking, separate the evidence types by what they can and cannot tell you.

• Guidelines synthesize evidence into recommendations for a defined context. They are useful for orientation and for identifying the questions a field considers settled, but their recommendations inherit the populations and settings they were written for.
• Systematic reviews assemble and appraise studies addressing a defined question. Cochrane's search and selection methodology illustrates the discipline involved: explicit eligibility criteria, multiple sources, staged screening, and linked reports.
• Trials test an intervention in a defined population under controlled conditions. They provide the primary comparative evidence but are bounded by their eligibility criteria and outcomes.
• Observational evidence describes associations in populations as they are actually managed. It can address questions trials do not, but its design limits what it can support.

Ranking these families is a starting heuristic, not a verdict. A systematic review of poor-quality or indirect studies does not automatically outrank a well-conducted trial that directly matches your question.

These four judgments are often collapsed into one label. Keep them distinct.

Evidence level is the design-based rank you assign — guideline, systematic review, trial, or observational study — before reading the details.

Applicability asks whether the evidence population, setting, and management context match yours. A trial in hospitalized patients is a different clinical task from outpatient management, even if the drug and outcome names are identical.

Indirectness asks whether the comparison, outcome definition, or population differs from your question. A review that pools mixed settings is indirect for a narrowly defined outpatient question.

Study limitations are internal: risk of bias, conduct problems, and reporting gaps within a study. A trial can be directly applicable yet still have serious limitations.

A useful discipline is to record all four columns for every included source. When a high-level source scores poorly on applicability or indirectness, your hierarchy should show that rather than hiding it behind the design label.

Insert explicit checkpoints so the hierarchy does not drift.

1. Eligibility checkpoint: Does the record meet the population, comparison, and outcome criteria you wrote in Step 1? If not, exclude it and record why.
2. Duplicate-report checkpoint: Cochrane recommends linking multiple reports of one study. Confirm you are not counting the same trial twice.
3. Design checkpoint: Classify each included record into guideline, systematic review, trial, or observational evidence.
4. Directness checkpoint: Mark whether the population and setting match your question, or whether the evidence is indirect.
5. Limitations checkpoint: Note conduct or reporting concerns before assigning a rank.

Documenting these decisions is part of the method, not an afterthought. Cochrane's guidance to document the search and selection process supports exactly this kind of audit trail.

Suppose your search returns a guideline covering respiratory infections, a systematic review pooling inpatient and outpatient studies, a trial restricted to outpatients, and an observational cohort from routine practice.

• The guideline orients you to the field's recommendations, but you check whether its recommendations address outpatients specifically.
• The systematic review ranks high by design, yet its pooled population is indirect for your outpatient question. Record that indirectness explicitly.
• The trial in outpatients is directly applicable to your population and comparison; check its limitations before treating it as decisive.
• The observational cohort may reflect routine management, but its design limits causal inference about the compared courses.

Your hierarchy might place the outpatient trial and the directly applicable portions of the review at the top for this question, with the guideline as context and the cohort as supporting information. The point is that the ranking follows the four separate judgments from Step 4 — not the design label alone.

Once sources are ranked, state what the hierarchy supports and what it does not. If the only directly applicable evidence is a single trial, say so. If the highest-ranked design is indirect for your population, describe the gap rather than borrowing its conclusion.

This is also where you decide whether the question is answerable with current evidence or needs to be narrowed. Narrowing the population or outcome often converts an indirect evidence base into a direct one.

For longer synthesis work, Scholar Agent supports literature review tasks, and Research Projects provides a workspace for organizing a project. When you need to keep the included papers together, My Library lets you save or upload papers and organize them, and Library QA lets you ask questions based on papers uploaded to My Library or added from search results.

Use these rules to decide when to stop building the hierarchy or escalate the question.

• Stop when every included source has been classified, checked for duplicate reports, and assessed on applicability, indirectness, and limitations.
• Escalate to a narrower question when the highest-ranked evidence is indirect for your population or outcome.
• Escalate to a broader search when eligibility screening leaves too few directly applicable sources; revisit subject headings and tree positions before concluding anything about the field.
• Rerun the search before finalizing, as Cochrane recommends, so the hierarchy reflects current records.
• Re-check citations before circulating a draft: TrueCite checks BibTeX against real academic databases to flag hallucinated references.