Network Meta-Analysis Assumptions: Transitivity, Consistency and Heterogeneity

By Dr. Esmaeel Saeedy Robat, Founder of MetaSyn Academy · Published meta-analyst (Nature Human Behaviour, 2026)

The hardest question in network meta-analysis is not whether every treatment is connected on a graph. It is whether the trials are comparable enough for the indirect links to mean what the model says they mean. A network can be dense, statistically elegant and still answer the wrong clinical question if the treatment comparisons differ systematically in variables that modify relative effects. That is why transitivity, heterogeneity and consistency have to be examined as distinct parts of one credibility argument rather than as three boxes to tick.^1,2

Network meta-analysis extends pairwise meta-analysis by combining direct evidence, indirect evidence and, where available, both together. This creates a richer comparative evidence structure, but it also creates assumptions that ordinary pairwise synthesis does not need in the same way. The most consequential is transitivity: the idea that different treatment comparisons are sufficiently comparable, especially with respect to treatment-effect modifiers, to support indirect inference. Consistency or coherence is related, but it is not the same thing. It is the degree of agreement between direct and indirect evidence where the network structure lets us compare them.^1,5,18

Connectivity is necessary, but it is not credibility

A treatment network is usually represented as nodes connected by edges. Nodes represent interventions or intervention classes; edges represent direct comparisons from eligible studies. If every treatment is reachable from every other treatment, the network is connected. That structural property is necessary for a conventional NMA to estimate all pairwise contrasts across the network. If the network breaks into separate components, some cross-component contrasts cannot be estimated without introducing a different method and additional assumptions.¹

That is the mathematical question. The clinical question comes next. A placebo-controlled trial of treatment A in newly diagnosed patients and a placebo-controlled trial of treatment B in heavily pretreated patients create a connected A–placebo–B path. They do not automatically create a credible indirect A-versus-B comparison. If prior treatment modifies relative treatment effects, the path is structurally available but clinically suspect. Network geometry therefore belongs at the beginning of appraisal, not at the end of it.

Open, star and disconnected networks are different problems

A star network has a central comparator, often placebo or usual care, with active treatments radiating outward and no closed loops. It can still be connected and therefore estimable. What it cannot provide is an ordinary empirical comparison of direct and indirect evidence between the active treatments, because there is no direct A-versus-B edge to compare with the indirect A–C–B route. In such a network, incoherence is better described as structurally unavailable or not assessable, not “absent.” Transitivity appraisal becomes more important because the network offers no direct-indirect diagnostic cross-check.^1,8

A disconnected network is different. If two groups of treatments have no path between them, a conventional NMA cannot estimate all cross-component contrasts under its ordinary evidence structure. Analysts can restrict inference to connected components or consider specialist population-adjustment/external-control approaches, but those methods bring new assumptions. Calling the original network “invalid” is too broad; the precise issue is that the desired cross-component contrast is not identified by the conventional connected network alone.

Transitivity is the central clinical-methodological assumption

Transitivity is often described through the hypothetical multi-arm trial idea: could the competing interventions reasonably have been randomized within the same target population and decision context? This is a useful starting point because it forces the analyst to think beyond the graph. Trials can share a comparator and still differ in treatment line, severity, eligibility, co-interventions, dose, follow-up, outcome definition or study era in ways that change relative treatment effects.^1,2

The most operational way to evaluate transitivity is to identify plausible treatment-effect modifiers before interpreting the network and compare their distributions across the sets of studies contributing to different treatment comparisons. The word plausible matters. A variable does not have to demonstrate a statistically significant interaction in the available trials before it deserves attention. Interaction tests are often underpowered, especially with aggregated trial-level data. Clinical knowledge, biological rationale, subgroup evidence, prior research, guideline knowledge and the HTA decision context can all justify prespecifying a potential modifier.^4,5

Prognostic factor is not another name for effect modifier

A prognostic variable changes baseline outcome risk. A treatment-effect modifier changes the relative effect of treatment. The two can overlap, but they are not interchangeable. This distinction becomes especially important when reviewers compile long baseline-characteristic tables and assume that every imbalance is equally threatening to transitivity. A difference in a strongly prognostic variable may matter for absolute risk while leaving a relative treatment effect stable; another variable may alter the relative effect directly. The appraisal should focus on clinically plausible modifiers while still documenting important uncertainty where the role of a variable is unclear.

Missing information is an uncertainty, not a reassuring result

Aggregate trial reports often omit variables needed for transitivity assessment. A reviewer may know that previous treatment is likely to modify effect but find it reported in only half of the studies. There is no defensible way to convert that absence into “balanced.” The safer conclusion is that transitivity is insufficiently assessable for that modifier. Empirical research supports this cautious stance: published NMAs continue to underreport transitivity assessments, and study-level statistical approaches for detecting intransitivity have limited performance.^3,4,17

Heterogeneity asks a different question

Heterogeneity is often discussed as though it were a single statistic. In practice there are at least three relevant layers. Clinical heterogeneity concerns differences in participants, interventions, outcomes or settings. Methodological heterogeneity concerns study design and conduct. Statistical heterogeneity is the residual variation in estimated effects beyond sampling error and is commonly summarized through a between-study variance such as τ², with I² sometimes reported for direct comparisons. These layers should be interpreted together.¹

In NMA, models often use a common heterogeneity variance across treatment comparisons because many comparisons are informed by too few studies to estimate separate variances reliably. That is a modeling assumption, not a clinical truth. The appraisal question is therefore not “Is τ² below the acceptable threshold?” No universal threshold exists. The question is whether the heterogeneity model is transparent, whether clinical and methodological sources were explored, how uncertain the estimate is, and whether the resulting spread changes the interpretation of the treatment effects.

Consistency and incoherence: what the data can test, and what they cannot

If transitivity is plausible, direct and indirect evidence are expected to agree apart from random variation and heterogeneity. Statistical disagreement is usually called inconsistency; CINeMA often uses the term incoherence. The terminology differs, but the methodological point is the same: where the network contains both direct and indirect evidence for a contrast, the analyst can examine whether those sources tell materially different stories.^6,8

Local methods examine particular contrasts or loops. Node splitting or side splitting separates direct evidence from the indirect evidence available elsewhere in the network for a selected comparison. Loop-specific approaches examine inconsistency within a closed loop. Global methods, such as the design-by-treatment interaction framework, ask whether inconsistency is evident across the network as a whole. These methods are complementary diagnostic perspectives. There is no universal rule that every NMA should start with one and then mechanically move to the other.^6,7

When incoherence is found

Detected disagreement should trigger investigation rather than automatic deletion of whichever study or comparison looks inconvenient. Plausible explanations include differences in effect modifiers, outcome timing, treatment definitions, study design or bias. Sensitivity analysis, revised node definitions, subgrouping or network meta-regression can sometimes clarify the source. But statistical adjustment cannot make fundamentally incomparable populations jointly randomizable after the fact. If a major clinical incompatibility remains, the appropriate conclusion may be to narrow the network or stop interpreting the affected indirect comparison as credible.

Bias in the NMA is broader than bias in its trials

Study-level risk-of-bias tools remain essential, but network meta-analysis creates additional ways in which the review itself can become biased. The 2025 RoB NMA tool was developed specifically to assess limitations in how an NMA was assembled, analyzed and interpreted. Its unit of assessment is the NMA review, not the individual randomized trial. ROB-MEN addresses a different construct: risk of bias due to missing evidence, including unavailable results and unpublished studies, and it is integrated with the reporting-bias domain of CINeMA.^9,10

These tools are complementary, not synonyms and not replacements for one another. An NMA can be competently modeled yet vulnerable to missing evidence; another may contain a fairly complete evidence base but be assembled or interpreted in a biased way. A practical assumptions appraisal should therefore ask whether these risks were examined, then hand off to the appropriate specialist framework instead of reproducing it.

Treatment rankings come after credibility, not before it

SUCRA values, P-scores, probabilities of being best and mean ranks can be useful summaries, but they answer ranking questions rather than clinical superiority questions. Two treatments can occupy different ranks while their estimated effects are clinically indistinguishable. Rankings can also differ depending on the metric used and how uncertainty is incorporated. The treatment-hierarchy question should therefore be explicit: what does “better” mean for this outcome and decision?^13,14

Recent work has proposed ranking methods that incorporate minimally important differences, which directly addresses the problem of treating trivial effect differences as meaningful separation. These methods are promising but should still be described as developing rather than as a universal replacement for established rank metrics.¹⁵

When should the NMA be reconsidered?

There is no single threshold that turns an NMA from acceptable to unacceptable. The better approach is to distinguish structural barriers from credibility concerns and potentially investigable issues.

Where CINeMA, GRADE and PRISMA-NMA fit

CINeMA evaluates confidence in network estimates across six domains: within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence. GRADE-NMA addresses closely related certainty questions using GRADE’s conceptual framework. The two approaches are not identical; empirical comparison has shown that they can produce different certainty judgments for the same network estimates. That is another reason not to treat transitivity or heterogeneity as mechanically scoreable.^8,16

For reporting, PRISMA-NMA 2015 remains the current published NMA extension at the evidence cut-off used for this guide. An update is underway. The 2025 scoping review supporting that update identified new candidate reporting needs, including clearer reporting of methods used to assess homogeneity and transitivity and developments in complex NMA methods. Until a replacement guideline is finalized, the published 2015 extension remains the operational baseline, used alongside PRISMA 2020 where appropriate.^11,12

Apply the reasoning before interpreting the result

The paired Network meta-analysis assumptions checklist turns the reasoning in this guide into a structured appraisal. Use it to record network geometry, effect-modifier comparability, heterogeneity, incoherence, bias and ranking concerns without calculating an overall validity score. The purpose is to make the reasoning auditable, especially where the evidence is incomplete or a specialist judgment is still needed.

For the larger comparative-effectiveness context, including how NMA relates to direct evidence, ITCs, population adjustment and HTA decision questions, use Evidence Synthesis for HTA and HEOR: Comparative Effectiveness and Decision Support. Pair 31 deliberately stops before the detailed appraisal of anchored and unanchored indirect comparisons, which belongs to the separate ITC guide.

Frequently asked questions

References