The Decision Literacy Hub
Systematic Review, Meta-Analysis and Evidence Synthesis Guides
Methodology guides for systematic review, meta-analysis, and evidence synthesis. Field manuals for researchers who refuse to guess. Written by Dr. Esmaeel Saeedy Robat, published meta-analyst (Nature Human Behaviour 2026).
Evidence synthesis is difficult because each decision changes what becomes defensible next
Evidence checked: 12 August 2026
A rigorous systematic review is not a sequence of clerical tasks. It is a chain of interdependent methodological decisions. The question determines what evidence is relevant; eligibility criteria determine what can enter the review; searching determines what can be found; appraisal affects how studies should be interpreted; and synthesis choices determine which conclusions the assembled evidence can reasonably support. A weakness introduced at one stage can therefore survive into every stage that follows.
This dependence is visible in the structure of major methodological handbooks. The Cochrane Handbook treats question formulation, eligibility, searching, data collection, risk-of-bias assessment, synthesis, heterogeneity and interpretation as connected parts of one review process rather than isolated technical exercises.1 JBI extends the same principle across multiple forms of evidence synthesis, recognizing that different questions require different review designs, evidence types and synthesis methods.2
The problem is not a shortage of guidance
Evidence synthesis is supported by an extensive methodological literature: handbooks for conducting reviews, standards for particular designs, reporting guidelines, appraisal instruments, statistical methods, certainty frameworks and specialist guidance. The difficulty for learners is that these resources answer different questions and operate at different levels. Knowing how to complete one procedure does not necessarily show when that procedure is appropriate, what assumptions it carries, or how its output should constrain the next decision.
Reporting guidance illustrates the distinction particularly well. PRISMA 2020 was developed to improve transparent reporting of why a systematic review was conducted, which methods were used and what results were found.3 It makes a completed review more inspectable. It is not a substitute for the methodological standards needed to design the review, conduct the analysis or judge whether the chosen approach was appropriate.
Professional competency frameworks reveal a related problem. Existing frameworks tend to be specific to particular roles or sectors rather than defining one universal evidence-synthesis curriculum. Townsend and colleagues, for example, developed a systematic-review competency framework specifically for librarians, while the CABI Evidence Synthesis Skills Framework was developed primarily for evidence-synthesis work in the agrifoods sector.45 These frameworks are useful within their intended contexts, but their specificity also illustrates why evidence-synthesis expertise cannot be reduced to a single generic checklist of skills.
A sequential knowledge base must still allow methodological branching
Review work has an underlying order, but it is not a rigid universal pipeline. A quantitative intervention review, scoping review, qualitative evidence synthesis, diagnostic review, network meta-analysis and overview of reviews do not make identical methodological decisions. The review question determines which branch of the evidence-synthesis system becomes relevant.
This creates an educational tension. Learners need enough sequence to understand dependency: why protocol decisions precede analysis, why searching precedes study selection, and why appraisal precedes confident interpretation. At the same time, they need enough branching to recognize that methods change when the question, evidence type or intended use changes. Teaching only a linear workflow risks implying that one procedure fits every review. Teaching isolated specialist methods risks the opposite problem: losing sight of how one methodological choice affects the larger synthesis.
Procedural knowledge
The researcher can carry out a defined task: construct a search, screen a record, complete an appraisal, calculate an effect estimate or populate a reporting item.
Methodological judgment
The researcher can explain why the task is appropriate, identify the assumptions being made, recognize when the method no longer fits, and understand how the decision affects the synthesis that follows.
Decision literacy connects methods that are otherwise learned separately
In this context, decision literacy can be understood as the ability to recognize a consequential methodological decision, identify the evidence or standard that governs it, compare defensible alternatives, make the underlying assumptions visible and document why one course of action was chosen. It also includes recognizing when a decision exceeds the review team’s competence and requires specialist input.
The principle extends beyond review conduct. GRADE Evidence-to-Decision frameworks were developed because synthesized evidence alone does not automatically determine a recommendation: evidence must be considered explicitly alongside other relevant judgments in a structured and transparent process.6 The same distinction is important earlier in evidence synthesis. Data, calculations, checklists and software can inform a decision; they do not remove the need to understand what is being decided.
Find the guidance where the decision occurs
Browse by the methodological or professional problem in front of you. Each collection brings related guidance together while preserving links to the surrounding stages of evidence synthesis.
Turn a research idea into a prespecified review plan. Define the question, eligibility, roles, timeline, registration strategy and transparent handling of later amendments.
Foundational guides
Translate a review question into reproducible information retrieval, database-specific syntax, transparent search records and defensible approaches to grey literature.
Foundational guides
Apply eligibility criteria consistently from deduplicated records to the final included-study set, document exclusion decisions and account for records transparently.
Foundational guides
Design and pilot extraction systems, build study-characteristics records, manage missing information and preserve an auditable path from the source report to the synthesis dataset.
Foundational guides
Match appraisal to the included evidence, support domain-level judgments with study information, resolve disagreement and carry bias judgments into synthesis and interpretation.
Foundational guides
Match data to compatible effect measures, choose a model whose assumptions fit the inferential goal and interpret pooled effects and forest plots without overstating what they establish.
Foundational guides
Interpret heterogeneity, prediction intervals and sensitivity analyses; investigate small-study effects without treating a diagnostic statistic as a complete explanation of the evidence.
Foundational guides
Assess certainty outcome by outcome, make judgments transparent, prepare Summary of Findings tables and understand how evidence assessment enters a broader evidence-to-decision process.
Foundational guides
Turn completed methods and results into a transparent manuscript, abstract and reporting record that lets readers see what was planned, what was done and how the conclusions were reached.
Foundational guides
Match the review design to the question and decision need. Compare purposes, eligibility logic, search expectations, synthesis approaches, reporting standards and updating models across major evidence-synthesis designs.
Foundational guides
Examine comparative-synthesis assumptions and plan evidence reviews for HTA, HEOR and market-access contexts where comparators, networks, decision scope and auditability matter.
Foundational guides
Select, appraise, extract and synthesize PROM evidence while keeping constructs, instruments, measurement properties, interpretability and feasibility conceptually distinct.
Foundational guides
From methodological capability to responsible professional practice
Develop the professional capabilities that sit around evidence-synthesis methods: role awareness, demonstrable competence, teaching and supervision, consulting, evidence leadership, decision support and accountable AI-enabled practice.
Methodological competence becomes professional practice when researchers take responsibility for work that others must be able to inspect, trust and use. That includes understanding the limits of one’s competence, knowing when specialist input is needed, communicating uncertainty, supervising others and remaining accountable when technology performs part of the workflow.
RECOMMENDED STARTING GUIDE
Researchers remain responsible for every AI-assisted decision
AUTOMATE?
VERIFY?
REPORT?
Keep humans responsible
AI for Systematic Reviews: What Researchers Can Automate, What They Must Verify, and What They Must Report
AI can assist with searching, screening, extraction, and reporting, but its reliability changes by task, tool, and review context. This evidence-based editorial explains how to define each use, validate consequential outputs, preserve an audit trail, and keep methodological decisions under qualified human control.
Dr. Esmaeel Saeedy Robat
Published meta-analyst · Evidence checked July 2026
New methodology guides per month · All written by a working meta-analyst
New methodology guides per month · All written by a working meta-analyst
Careers in Evidence Synthesis: Roles, Skills and Professional Pathways
How to Choose a Patient-Reported Outcome Measure for Evidence Synthesis
How to Extract Measurement-Property Data for PROMs in Systematic Reviews
How to Use the COSMIN Risk of Bias Checklist
How to Plan a Systematic Literature Review for HEOR and HTA
Indirect Treatment Comparisons Explained: Common Comparators, Assumptions and Credibility
Methodological competence depends on seeing the connections between decisions
The complexity of evidence synthesis comes partly from the number of methods involved, but more importantly from their dependence on one another. A review can execute each individual task competently and still reach a weak conclusion if the tasks were connected by inappropriate assumptions. A comprehensive search cannot repair an ill-defined eligibility framework. Sophisticated meta-analysis cannot compensate for incompatible studies. Transparent reporting cannot retrospectively correct a decision that was methodologically indefensible.
For this reason, evidence-synthesis education should not be understood as the accumulation of isolated technical skills. Reviewers need to understand both the local procedure and its position in the wider inferential chain. This includes knowing which earlier decisions a method depends on, which later conclusions it constrains, and when new information requires an earlier decision to be reconsidered. The review process is sequential, but expert practice is also recursive.
Prespecification provides one example. Protocol guidance such as PRISMA-P encourages reviewers to define important methods before the results of included studies are known.7 Yet prespecification does not eliminate judgment. Unexpected study designs, unavailable outcomes or previously unforeseen analytical problems may require a justified amendment. Methodological rigor lies not in pretending that adaptation never occurs, but in distinguishing principled adaptation from undisclosed data-driven decision making.
Critical appraisal creates a similar distinction. Instruments such as AMSTAR 2 structure appraisal of systematic reviews, but the instrument does not remove the need to understand why particular weaknesses are important or how they should affect interpretation.8 The same principle applies to risk-of-bias tools, certainty frameworks, statistical diagnostics and reporting checklists: structure can make judgment more transparent, but structure does not make judgment unnecessary.
Sequence
Understand which decisions must precede others and how errors or assumptions can propagate through the review.
Branching
Recognize when the question, design, evidence type or intended use requires a different methodological path rather than the default procedure.
Integration
Connect searching, appraisal, synthesis, certainty, reporting and interpretation so that the final conclusion remains traceable to the decisions that produced it.
A knowledge base should reduce fragmentation without creating false simplicity
Fragmentation is not solved by placing every method into one linear checklist. Evidence synthesis contains genuine specialization. Information retrieval, advanced statistical synthesis, qualitative methods, measurement science, HTA, certainty assessment and professional AI governance each require forms of expertise that may exceed the competence of a generalist reviewer. Existing competency frameworks reflect this specialization by defining capability within particular professional or sectoral contexts rather than asserting one universal profile.45
The educational objective is therefore not to make every researcher expert in every technique. It is to make the larger decision system visible. A competent reviewer should be able to locate the problem, identify the governing methodological standard, understand what the method can and cannot establish, recognize when specialist competence is required, and preserve enough documentation for another researcher to reconstruct the reasoning.
Synthesis and decision making should remain connected but distinct
A systematic review aims to assemble and evaluate evidence systematically. A meta-analysis may estimate a pooled effect. Certainty assessment can characterize confidence in an evidence body. None of these steps, by itself, determines what a clinician, policy maker, institution or individual should do. GRADE Evidence-to-Decision frameworks make this boundary explicit by placing research evidence within a broader and transparent process that can also consider benefits and harms, values, resources, equity, acceptability and feasibility.6
Preserving that boundary is part of methodological maturity. Evidence synthesis should make the evidence clearer and the uncertainty more visible. It should not hide value judgments inside statistical output or convert a methodological result into a recommendation without showing the additional reasoning that connects the two.
Methodological boundary: Educational guidance, templates, software and decision aids can structure evidence-synthesis work, but none can establish the validity of a review independently of its question, data and context. Reviewers remain responsible for checking the standards that apply to their design, registry, discipline and publication setting, and for seeking specialist methodological input when the required judgment exceeds the competence of the review team.
Frequently asked questions
These answers describe how the methodology guides can be used alongside practical resources, specialist standards and structured teaching.
Are the methodology guides free to read?
Yes. Guides in the Decision Literacy Hub can be read without course enrollment. Some complementary downloadable resources may ask for an email address so MetaSyn Academy can send the editable file and relevant methodology updates. Courses and certification remain separate educational products.
How are MetaSyn methodology guides researched and checked?
Each guide is researched against sources appropriate to its methodological question, including current methods handbooks, official reporting or instrument guidance and primary methodological research where relevant. Consequential claims, references, methodological boundaries and practical implications are checked before publication, with final academic approval by Dr. Esmaeel Saeedy Robat.
Should a reporting checklist be used as a guide to conducting the review?
Not by itself. Reporting guidelines such as PRISMA improve transparency about what was done and found, but conduct decisions should be based on the methodological guidance appropriate to the review design and question. Reporting standards and conduct standards are related, but they solve different problems.
Where can I find the matching template, worksheet or tool?
Published guides link contextually to relevant practical resources. You can also browse the Free Systematic Review & Meta-Analysis Templates library or use the resource links within the methodological collections above.
Are all evidence-synthesis methods applicable across research fields?
No. Many principles of transparent review conduct transfer across disciplines, but review designs, terminology, appraisal methods, synthesis techniques and reporting requirements can be context-specific. Specialist areas such as HEOR, network meta-analysis, GRADE and COSMIN require their own methodological standards and should not be generalized automatically beyond their intended scope.
Where does AI fit within evidence-synthesis methodology?
AI can support parts of evidence-synthesis work, but its methodological acceptability depends on the task, evidence for performance, validation and verification arrangements, data governance and the consequences of error. For workflow-level guidance, see AI for Systematic Reviews . Professional questions about capability, accountability, supervision and human judgment are addressed in Professional Practice in Evidence Synthesis .
How do the guides connect with structured training?
Individual guides address defined methodological decisions. Structured training is useful when a researcher needs to develop the dependencies between those decisions across an entire review. Protocol-focused learners can begin with How to Write a Systematic Review & Meta-Analysis Protocol , while the Meta-Journey curriculum provides the wider taught pathway.
HOW THIS CONNECTS TO THE ACADEMY
From free methodology guides to structured training and graduate collaboration
Use the free guides to understand methodological decisions, the paired templates and tools to apply them, and the Meta-Journey for structured teaching, feedback, and assessment.
Methodology Guides
Read free, field-agnostic guides to systematic review, meta-analysis, and evidence synthesis. Use each guide with its paired practical resource.
The Launchpad
Develop and register a PRISMA-P-aligned protocol for your own systematic review through structured teaching and founder feedback.
The 13-Course Meta-Journey
Build systematic review, meta-analysis, and evidence synthesis skills across 13 courses and four progressive phases.
The Summit Circle
Continue your work through advanced discussion, scholarly collaboration, and professional development with MetaSyn graduates.
Receive new methodology guides and practical resources when they are published.
One email per month when a new guide publishes. Plus templates and methodology updates from the Resource Infrastructure. No marketing noise, no sales pitches, only methodology.
References
These sources support the general argument on methodological interdependence, reporting, competency and evidence-to-decision reasoning. Individual guides carry their own topic-specific evidence bases.
- Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5, updated August 2024. Cochrane; 2024. Available from: Cochrane Handbook, current online edition .
- Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z, editors. JBI Manual for Evidence Synthesis. JBI; 2024. doi:10.46658/JBIMES-24-01 .
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71 .
- Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, Smith JE. A competency framework for librarians involved in systematic reviews. J Med Libr Assoc. 2017;105(3):268-275. doi:10.5195/jmla.2017.189 .
- CABI. Evidence Synthesis Skills Framework. Wallingford: CABI; 2025. Available from: Evidence Synthesis Skills Framework .
- Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016. doi:10.1136/bmj.i2016 .
- Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. doi:10.1186/2046-4053-4-1 .
- Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. doi:10.1136/bmj.j4008 .