The problem with most evidence

Most published research, taken in isolation, is unreliable as a basis for decisions. This is not an accusation against any individual study. It is a structural fact of how evidence accumulates. One trial may be underpowered. Another may be context-specific. A third may be an outlier driven by sampling error, publication bias, or methodological choices made before the data were collected. Any single study can be wrong, and we usually do not know in advance which one.

The most common alternative, namely the traditional narrative literature review (or worse, citing one or two convenient studies), does not solve this problem. It compounds it. A narrative review selects which studies to discuss; a systematic review specifies the criteria for selection before the studies are screened, applies them transparently, and reports what was found regardless of whether the findings were convenient.

This is the methodological gap Station 1.1 exists to close. Before we discuss how to conduct a systematic review (Stations 1.2 through 1.5), we have to be clear about why the methodology exists at all, and why your research question deserves it.

The three core synthesis methods

Evidence synthesis is not a single method. It is a family of methods, each with a specific job. The most common three, the ones this course returns to repeatedly, are these:

You will work with all three across Course 1 and Course 2. The framework for choosing which method fits your question is taught in Station 1.3, when you build your protocol around one of the six question frameworks (PICO, PECO, SPIDER, ECLIPSE, PCC, PIRT).

Where synthesis sits in the evidence hierarchy

The classical evidence pyramid orders study designs by their methodological capacity to isolate cause from correlation. At the top sits systematic review and meta-analysis; at the base sits expert opinion. Between them, in ascending order of methodological rigour:

This pyramid is a teaching scaffold, not an absolute truth. Modern methodologists, most influentially Murad and colleagues, have argued the pyramid oversimplifies appraisal, proposing instead that systematic review functions as a lens through which all designs are interpreted, rather than a tier above them. The academy teaches the pyramid first, then progressively builds toward the more nuanced view. For now, hold the pyramid as a starting point.

What this means for you, now

You are about to spend Course 1 building a protocol for a systematic review on your own research question. Before you do, three implications follow from everything above.

First, your protocol matters more than your data. The choices you make before you see the data (eligibility criteria, search strategy, outcome definitions, synthesis plan) are the methodological commitments that protect your conclusions from your own preferences. Station 1.2 explores why protocols are not bureaucratic documents but professional commitments. Station 1.3 walks you through writing one against the PRISMA-P 2015 standard.

Second, the gap your review fills must be specific and defensible. “There is no review on this topic” is rarely true and never sufficient. The honest position is: “There is no review that has done this specific synthesis with these criteria over this time frame, and that gap matters because [defensible reason].” If you cannot finish that sentence, your protocol needs more thought before you proceed.

Third, methodology serves the question, not the other way around. The framework you choose (PICO, PECO, SPIDER, ECLIPSE, PCC, PIRT), the synthesis method (systematic, meta-analytic, scoping), and the appraisal tool (RoB 2, ROBINS-I, AMSTAR 2) all follow from what you are trying to learn. Method-first thinking (choosing meta-analysis because it sounds prestigious, or scoping review because it sounds easier) leads to research that answers the wrong question with the wrong tool.

The remainder of Course 1 will hold you to these three commitments. By Station 1.5, you will have a registered protocol on PROSPERO or OSF that defends its question, its method, and its position in the existing evidence base.