How do I fix a reporting checklist used as a quality appraisal tool?

Separate the reporting items from the appraisal domains, reapply a design-specific appraisal tool, and revise any quality claims that relied on checklist scores. Reporting guidelines such as CONSORT 2025 and STROBE assess how completely a study is reported; they do not judge study quality. The NHLBI Study Quality Assessment Tools focus reviewers on internal validity and potential flaws in study methods or implementation, and users must define their own judgement parameters.

1. Sort each checklist entry by purpose. Mark items that ask whether a detail was reported (for example, whether allocation concealment was described) as reporting-completeness items. Move any item that asks you to judge risk of bias or methodological soundness into a separate appraisal column. Expected result: two distinct lists with no item serving both purposes.
2. Select an appraisal instrument matched to the study design. Use the NHLBI design-specific tool for the design under review and set your own judgement parameters before scoring. Expected result: a documented appraisal instrument and decision rules that you can apply consistently across studies.
3. Reapply the correct instrument to each purpose. Score reporting completeness with the reporting checklist and score internal validity with the appraisal tool. Expected result: separate scores that you can trace back to the instrument that produced them.
4. Revise quality claims that relied on checklist scores. Find every sentence in your review that treats a reporting score as evidence of study quality, and rewrite it to state what the reporting checklist actually showed or replace it with the appraisal result. Expected result: quality statements supported by the appraisal tool.
5. Document the separation in your methods. Record which instrument you used for reporting and which for appraisal, and note the change if you corrected an earlier draft. Expected result: readers can see that each instrument was applied to its intended purpose.

A common failure mode is a single composite score that mixes reported details with judgements about bias. Keep the two outputs separate in your tables so a reader can see reporting completeness and internal validity as distinct findings.