Systematic Review Screening, Study Selection and PRISMA Flow Diagram

Inclusion is produced by a chain of reviewable decisions

Study selection converts database search hits into the final set of synthesized studies. Each inclusion or exclusion label relies on multiple shifts in analysis unit, judgment under incomplete reporting, and a clear audit trail from the initial search record to the final publication.²

The screening pipeline starts with deduplicated records. Reviewers screen titles and abstracts, retain potentially eligible items, request full texts, track retrieval failures, evaluate reports against operational criteria, and link multiple publications originating from the same study.^2,4 PRISMA 2020 provides a structured format to report these decisions, but it remains a reporting guideline rather than a conduct manual. Methodological frameworks such as Cochrane and JBI set their own requirements for executing and documenting this process.^4,6

Flawed selection strategies introduce bias long before synthesis or risk-of-bias appraisals begin. Premature screening exclusions can drop a study’s sole report, unlinked secondary papers can lead to double counting, and vague exclusion justifications weaken reproducibility.⁴ High-quality screening aims for reproducible judgment: independent reviewers should be able to trace which criterion was applied, what data was available, who made the call, and how consensus was reached.

The unit changes from record to report to study

PRISMA 2020 separates a record (such as a database title and abstract) from a report (a journal article, preprint, or clinical trial register entry) and the underlying study itself.² This distinction directly dictates how items move through screening, where exclusion reasons apply, and why final report counts may not equal study totals.

Title and abstract screening evaluates records. Full-text appraisal evaluates reports. Final inclusion applies to studies, even when relevant data spans several reports. Cochrane requires grouping companion reports before synthesis to avoid analyzing multiple papers from a single trial as independent studies.^4,5 Robust screening management systems rely on unique record, report, and study IDs to maintain these relationships.

This structural clarity also prevents categorizing unretrieved papers as excluded. If a full text cannot be obtained, it was never evaluated against eligibility criteria; it belongs in the “reports not retrieved” section of the PRISMA flow diagram.³ Similarly, ongoing studies belong in designated tracking categories rather than forced exclusion lists.

Protocol criteria must become observable screening rules

Protocol criteria often require operational refinement before screening begins. Broad eligibility statements like “adults with chronic disease” or “adequate follow-up” need concrete indicators, acceptable data sources, and explicit rules for missing details.⁴ Defining these parameters upfront standardizes decision-making across the research team.

Operational rules clarify what constitutes evidence for inclusion, grounds for exclusion, or insufficient reporting. During title and abstract screening, missing details rarely justify immediate exclusion. Most protocols use over-inclusive initial screens and defer definitive eligibility decisions to full-text reviews.⁴ The “unclear” tag serves as a temporary hold for missing information, not a substitute for applying criteria.

Piloting forms on a small sample exposes ambiguous instructions early. Reviewers screen a test batch independently, compare criterion-level decisions, resolve discrepancies, and finalize the screening form before launching the main search.⁸ Testing continues until reviewers apply the criteria consistently across complex reports.

Independence protects the decision; disagreement reveals the rule

Governance standards for screening vary across evidence synthesis organizations. Cochrane intervention reviews require two independent reviewers for final eligibility decisions, while dual screening at the title and abstract stage is recommended.⁵ Frameworks like JBI or rapid review methodologies may apply alternative dual-screening or verification approaches, provided all deviations are documented.⁶

Single-reviewer screening carries measurable risk. Waffenschmidt and colleagues demonstrated that single screening misses more eligible studies than dual screening, with error rates varying widely based on topic complexity and reviewer experience.⁷ This variance underscores why review teams cannot assume a single-screening approach is sufficient without validation.

Dual screening requires preserving initial decisions. Overwriting individual screening decisions with final consensus votes removes valuable data about criteria clarity. Screening platforms should log initial reviewer votes, track disputed criteria, and record consensus rationale. Disagreements often highlight ambiguous criteria that require team clarification.

Inter-rater agreement measures like Cohen’s kappa summarize overall consistency, but they do not guarantee screening accuracy. Kappa values depend heavily on category prevalence and distribution.¹³ Analyzing specific criterion-level disagreements provides more actionable feedback for refining screening rules than relying on a single summary statistic.

One decision record must serve conduct, reconciliation and reporting

Screening logs, exclusion tables, study-report maps, and flow diagrams represent different views of a single selection ledger. Shared identifiers ensure counts match across these outputs. The primary screening ledger tracks decisions, full-text logs capture exclusion reasons, and study-report maps link multi-publication trials.

Documenting rule changes during screening preserves transparency. Unanticipated study designs, mixed population cohorts, or ambiguous trial reports often require criterion clarifications.⁴ Review teams should record the rationale and approval date for any rule modification and re-evaluate previously screened records against the updated standard.

Machine learning and text-mining tools assist in prioritizing records, deduplication, and preliminary screening, but performance varies by model and search volume.^9,10 While algorithmic stopping rules can reduce workload, current guidance requires review teams to maintain accountability for final inclusion decisions and study retrieval completeness.^11,12