An evidence matrix is the working table that turns a pile of search results into a defensible review. Each row is one study or one report; each column is a decision you will need to defend later — why it was eligible, what it found, how strong the design was, and where the full text lives. Build it early, while you are still searching, and it becomes the audit trail behind your manuscript, your coauthor handoff, and your response to reviewers.
The practical sequence below starts from a realistic starting state: a research question, a rough set of terms, and a folder of PDFs with inconsistent names. It moves through defining eligibility, running and documenting searches, screening in stages, extracting data, and checking readiness. A worked example and a reusable checklist keep the method reproducible.
What does an evidence matrix actually need to do?
Before you open a spreadsheet, decide what the matrix is for. In a literature review it serves three jobs at once:
- Record eligibility decisions. Cochrane's guidance on searching and selecting studies begins with eligibility criteria and treats selection as a documented, staged process rather than an informal sort.
- Hold the extracted evidence. Each included study needs a consistent set of fields so that synthesis and writing draw on the same values.
- Preserve the method trail. Search terms, sources, dates, and screening decisions should be recoverable so a coauthor or reviewer can retrace your steps.
A matrix that only lists titles and authors fails all three. A matrix that carries eligibility, extraction, and provenance columns can support the manuscript, the coauthor review, and a later update.
Step 1: Fix the question and eligibility criteria before searching
Cochrane's recommendation is to plan searches from the eligibility criteria. That ordering matters: if you search first and define inclusion afterward, your matrix will encode post-hoc decisions that are hard to defend.
Write down, in plain sentences:
- Population or phenomenon the review addresses.
- Study designs you will include and exclude.
- Date and language limits, with a reason for each.
- Outcome or evidence types that count as relevant.
These four items become the first columns of your matrix and the anchor for every screening decision. Keep them in a short protocol note stored beside the matrix so the criteria and the table never drift apart.
Checkpoint: a coauthor can read your criteria and predict, for a sample of five papers, whether each would be included. If not, tighten the wording before you search.
Step 2: Run searches you can reproduce
Cochrane recommends using multiple sources, combining free-text terms with subject headings, and documenting the process. PubMed supports field tags, phrase searching, filters, Advanced Search, and saved searches, and its search details show how terms were translated and combined — which is exactly the record you want to keep. Google Scholar documents searching by author and title, date limits and sorting, Cited by and Related articles, citation export, and email alerts, all of which help you extend and monitor a search.
For each source, log:
- the exact query string, including field tags and filters;
- the database or interface used;
- the date the search was run;
- the number of records returned.
If you use WisPaper's scholarly search to find academic papers, capture the same details in your log so the query can be rerun. A public entry point for finding papers is available through WisPaper Scholar Search, and the working search interface is at WisPaper search.
Checkpoint: rerun one saved query and confirm you get a comparable result set. If the numbers diverge wildly, your log is missing a filter.
Step 3: Deduplicate and screen in stages
Cochrane's process includes deduplicating records, screening in stages, and linking multiple reports of one study. Build the matrix to mirror those stages rather than collapsing them.
A staged screen looks like this:
- Deduplicate by DOI or title, keeping one record per work and noting duplicates.
- Title and abstract screen against your eligibility criteria, recording include, exclude, or uncertain with a one-line reason.
- Full-text screen for the uncertain and included records, recording the final decision.
- Link reports so that a study published as a conference paper and a journal article occupies one row with both reports noted.
Screening reasons are the most valuable column in the matrix. They let you report exclusion counts accurately and answer a reviewer who asks why a familiar paper is missing.
Step 4: Extract data into a consistent schema
Extraction is where matrices usually break down: one person records effect sizes as text, another as numbers, a third leaves the field blank. Fix the schema before extraction and pilot it on three papers.
A workable extraction set includes:
- citation and identifier (DOI where available);
- study design and sample;
- intervention or exposure and comparator;
- key outcomes with units;
- limitations the authors state;
- the page or section where each value was found.
The last field is what makes the matrix auditable. When a coauthor questions a number, you can point to the source location instead of re-reading the whole paper.
If you are organizing the underlying PDFs, save or upload papers and organize them in My Library so the matrix rows and the stored files share a stable reference. When you need to ask questions based on papers uploaded to My Library or added from search results, Library QA supports that step without leaving your collected set.
Checkpoint: a second reader extracts the same three papers independently and the two versions agree on every field, or the disagreements are resolved and the schema clarified.
Worked example: a manuscript revision with a coauthor handoff
Suppose you are revising a review manuscript and a coauthor will take over synthesis while you are away. The matrix is the handoff artifact.
- Criteria note. You write the four eligibility items and store them with the matrix.
- Search log. You record each query, source, date, and count, including the field tags and filters used.
- Screening sheet. You deduplicate, screen titles and abstracts, then full texts, recording a reason for every exclusion and linking multiple reports of the same study into one row.
- Extraction sheet. You fill the agreed schema and cite the page for each value.
- Synthesis notes. You add a column for how each study bears on your argument, so the coauthor can draft from the matrix rather than from memory.
When the coauthor opens the file, they can see why each paper is in or out, where each number came from, and which searches produced the set. That is the difference between a handoff and a scavenger hunt.
For a longer revision task, Scholar Agent fits literature review and related research work, and Research Projects gives the review a project home where the work can be organized.
Step 5: Verify references before they reach the manuscript
A matrix is only as good as its citations. Before references move into the manuscript, check that each entry corresponds to a real publication. TrueCite checks BibTeX against real academic databases to flag hallucinated references — a useful gate when a matrix has been assembled from many sources and exports.
Run this check after extraction and again after any bulk edit to the reference list. Record the date of the check in your method log so the verification step is part of the trail.
A reusable matrix template and checklist
Use this column set as a starting point and delete what your review does not need.
| Column | Purpose |
|---|---|
| Record ID | Stable identifier for the row |
| Citation | Authors, year, title, venue |
| DOI / identifier | Deduplication and lookup |
| Source and search date | Provenance |
| Eligibility decision | Include, exclude, uncertain |
| Exclusion reason | One line, tied to criteria |
| Linked reports | Other versions of the same study |
| Design and sample | Appraisal context |
| Key findings | Extracted values with units |
| Source location | Page or section for each value |
| Author-stated limitations | For synthesis and discussion |
| Synthesis note | How the study bears on the argument |
Pair the table with this checklist:
- Eligibility criteria written and stored with the matrix
- Every search logged with query, source, date, and count
- Duplicates resolved and multiple reports linked
- Screening reasons recorded for every exclusion
- Extraction schema piloted on three papers
- Source location recorded for each extracted value
- References verified before manuscript insertion
- Searches rerun before submission, per Cochrane's guidance
How do you know the matrix is ready?
Readiness is a set of observable conditions, not a feeling. The matrix is ready when:
- Every included row traces to a logged search. No paper appears without a source and date.
- Every excluded row has a reason that maps to a written criterion.
- Every extracted value has a location. A coauthor can verify any number without asking you.
- Multiple reports are linked, so one study does not inflate your count.
- References are verified and the check is dated in the log.
- A rerun of the searches produces a result set you can reconcile with the matrix.
If any condition fails, the gap tells you which step to revisit. That is the advantage of building the matrix as a method trail rather than a final table.




