How to Set Up a Systematic Review Project Before the Search Begins

Eight parts of a defensible project setup

These eight parts are not independent checkboxes. A change in the question can ripple into eligibility, search concepts, extraction fields, and synthesis all at once. Record each decision somewhere the rest of the team can actually see it, not just in one person’s notebook.

Confirm the need before fixing the question

Start with the decision the review is actually meant to support. A clinical team might need an estimate of benefit and harm. A policy group might need an account of implementation barriers. An education researcher might just need to map how a concept has been measured across the field. Each of those uses points toward a different review question and a different form of synthesis.

Before committing to the project, search for existing and ongoing reviews on the same topic. Check bibliographic databases, relevant repositories, and registries, and write down the date, the sources checked, and your judgement. A related review doesn’t automatically kill the project. If your population, context, outcome, method, or currency creates a real gap, the project still stands. What it does require is an honest, explicit account of what makes it different.

Choose the review type, then frame the question

The review type should follow the purpose, not the other way around. A tightly focused question about effects usually points toward an intervention review. A broader question about concepts and evidence gaps points toward a scoping review. Questions about lived experience, diagnostic accuracy, prevalence, prognosis, or implementation each need methods built for that specific purpose. Munn and colleagues lay out useful decision guidance for choosing between systematic and scoping reviews3.

Use a question framework that actually matches the purpose. PICO works well for many intervention questions. PCC can support scoping review questions. SPIDER helps structure some qualitative evidence syntheses. None of these frameworks decide eligibility or synthesis on the team’s behalf; they’re drafting aids, nothing more.

Write the unit of interest directly into the question. Reviews include studies, but a search retrieves reports, and one study can easily produce several papers, abstracts, registry entries, or data records. Planning for that distinction before screening even starts prevents double counting later and gives the data model a stable centre to build around4.

Build the team around the work

A systematic review needs both subject knowledge and method expertise, plus enough hands on deck for independent judgements, data checks, synthesis, and reporting. Cochrane advises teams to weigh expertise, time, resources, authorship, and conflicts right from the planning stage5.

Assign roles to tasks, not job titles. One person can wear several hats, but every task still needs a named owner and a backup. Decide who can approve changes to eligibility, outcomes, or analysis, and write down how the team will resolve screening and appraisal disagreements before they happen. For intervention reviews, Cochrane requires at least two people to make independent final eligibility decisions, and expects the team to define the resolution process in advance4.

Write the protocol and choose its public record

The protocol locks in the question and planned methods before any study findings can quietly steer them off course. It should cover the rationale, objectives, eligibility, information sources, search approach, selection, data collection, outcomes, appraisal, synthesis, meta-biases, and confidence in the evidence, wherever these elements apply. PRISMA-P spells out the reporting items readers expect to see in a systematic review protocol26.

PRISMA-P is good for complete reporting, but it won’t tell you which model, appraisal tool, search sources, or synthesis method actually fits your question. That decision comes from the relevant handbook, the review-type manual, statistical guidance, and whatever standard your field follows.

Pick the registration or publication route before the review moves forward. PROSPERO’s scope and timing rules shift from time to time, so check its current guidance directly7. OSF Registries offers a Generalized Systematic Review registration that covers systematic reviews, scoping reviews, and meta-analyses8. A journal protocol or another field-specific registry might fit better for some projects. Whatever you choose, write down why.

Turn eligibility criteria into operational rules

Eligibility criteria need to guide real decisions, not just sound reasonable on paper. “Adults,” “community setting,” “usual care,” and “long-term outcome” all leave room for two reviewers to read them completely differently. Nail down age thresholds, mixed samples, setting boundaries, acceptable comparators, outcome constructs, time windows, publication forms, language policies, and study designs.

Write rules for the awkward cases before the search starts: companion reports, conference abstracts, theses, preprints, overlapping samples, multi-arm studies, multiple publications from one study, and records with information missing. The protocol should say how the team will seek clarification, and when uncertainty should lead to exclusion versus temporary retention.

Pilot the criteria on a small set of known eligible, ineligible, and genuinely uncertain reports. Have reviewers decide independently, without discussion first, then compare reasons and revise the wording together. The goal is shared interpretation, not a flattering agreement number.

Design the data structure before selecting software

Software should support the review’s data structure, not define it. Figure out the structure first. At minimum, distinguish search records, unique reports, unique studies, study arms or groups, outcomes, time points, effect estimates, appraisal judgements, and synthesis datasets. Give everything persistent internal identifiers so a single value in a result table can always be traced back to a study and its source report.

The C5-DM framework describes literature-review data through conceptualization, collection, curation, control, and consumption, and it puts real weight on provenance, transparent collaboration, data quality, and structures that actually fit the review type9. Those principles hold whether the team uses specialist review software, spreadsheets, code, or some mix of all three.

Create separate states for source exports, deduplicated records, screening decisions, extracted data, analysis-ready data, and outputs. Leave the source material untouched, and record every transformation along the way. Broman and Woo offer practical spreadsheet principles like consistent rectangular data, one value per cell, and a proper data dictionary10. Wilson and colleagues add useful guidance on file organization, documentation, backups, and reproducible workflows11.

Plan extraction and synthesis before the search

The review question should point straight through to the planned output. Define the outcomes or concepts, measurement instruments, time points, comparisons, effect measures, subgroup variables, and sources of heterogeneity that actually matter. State whether a quantitative synthesis is expected, and which conditions would make pooling a bad idea. For qualitative or mixed-methods work, define the planned synthesis approach and the form of source data it needs.

Draft the extraction form directly from the planned tables and analyses. Every field should earn its place. The data dictionary should define allowed values, missing-data codes, units, and transformations. Pilot the form on studies that represent the full range of expected designs; if a form can’t represent one complex eligible study, it needs revision before full extraction begins.

Plan appraisal at whatever result level the review requires. Pick a tool that matches the study design and the claim under review, name who will make judgements, decide how independence will actually work, and settle how disagreements get resolved. Tool selection belongs in the protocol, since appraisal findings feed directly into synthesis and confidence in the conclusions.

Prepare the search without running the final search

Eligibility criteria give the search its conceptual basis. Identify which concepts should appear in the query and which criteria should instead be handled during screening. Choose databases and other sources based on their coverage, not just because the team already happens to have access to them. Plan citation searching, registry searching, grey-literature sources, author contact, and search updates wherever the review question calls for them.

Bring in an information specialist from the protocol-development stage, not after the fact. Cochrane recommends working with an experienced librarian or information specialist throughout the search process4. PRESS offers an evidence-based framework for peer review of electronic search strategies, covering question translation, Boolean and proximity operators, subject headings, text words, syntax, limits, and database adaptation12.

Write the search documentation plan before retrieval starts. PRISMA-S asks authors to report databases and platforms, full strategies, dates, limits, deduplication, supplementary methods, and search updates13. Decide now where the exact search strings, export files, counts, and deduplication records will actually live.

A systematic review is ready to search when the team can pass these checks

A launch gate turns preparation into one explicit team decision rather than a vague sense that things are probably fine. It doesn’t certify the quality of the whole review. It simply confirms that the search can begin without unresolved project defects that would otherwise force silent changes later on.

• The decision use, audience, and contribution are recorded.
• The review type and question framework fit the purpose.
• The protocol names the methods that govern each review stage.
• The registry or protocol-publication route has been checked.
• Eligibility criteria include operational rules for difficult cases.
• Reviewers have piloted eligibility and resolved rule differences.
• Roles, independent checks, conflicts, and approvals have owners.
• The report-to-study relationship has a defined identifier system.
• The extraction structure connects to the planned synthesis.
• Appraisal methods fit the included designs and intended claims.
• Sources and supplementary search methods have a rationale.
• The draft search strategy has an expert review plan.
• Raw files, working files, outputs, and backups have separate homes.
• A decision log and amendment process are open from day one.

Problems that become expensive after searching

The infrastructure is field-agnostic; the methods are not

Health, education, policy, environmental science, business, and applied linguistics all use different databases, evidence types, appraisal tools, and synthesis traditions. What they share is the need for a pre-specified question, explicit eligibility, accountable decisions, stable data, and an audit trail that actually holds up.

Use the common infrastructure to keep the work traceable, and lean on field and review-type guidance to choose the actual methods. Keeping these two things separate avoids two common mistakes: forcing a health-intervention workflow onto every question, and treating transparency as optional the moment you step outside medicine.

Related protocol-planning resources

Frequently asked questions