Portfolio · Project 09 · Evidence synthesis · October 2026
Comparing medicines that never met
Most new medicines are tested against one comparator, but payers and HTA bodies want to know how they compare with everything else in use. When no trial put the two side by side, analysts build an indirect treatment comparison. This page works through the family of methods, from the tempting shortcut that should never be used to the population-adjusted methods that HTA bodies now scrutinise most closely.
Two trials, one shared comparator
Change the trials
The Bucher indirect comparison
Bucher and colleagues (1997) showed that if two trials share a comparator, the effect of Drug Z against Immunotherapy B can be estimated from the two effects against Chemo A, keeping each trial’s randomisation intact. On the log scale the effects subtract and their variances add.
Enter the two trial results
Adding a small head-to-head trial closes a loop
Suppose a small trial, ZB-2, did compare Drug Z with Immunotherapy B. Now there is direct evidence and indirect evidence through Chemo A. A network meta-analysis combines both, weighting each by its precision, and the closed loop lets us test whether they agree. ZEN-1 and B-301 keep the values set in Part 2.
Change the direct trial
Reweighting ZEN-1 to look like B-301
Drug Z works better in marker M-positive patients (true hazard ratio 0.50, against 0.85 in M-negative), so marker M is an effect modifier. Performance status (ECOG 1) only changes prognosis. We hold individual patient data from ZEN-1 (400 patients) but only published summaries from B-301.
Describe B-301’s population
MAIC reweights the patients; STC instead fits an outcome model with the effect modifier and predicts the result in B-301’s mix. Both give an answer only for B-301’s population. ML-NMR generalises the idea across a whole network and any target population, but the 2024 EU guideline notes it cannot yet be applied to time-to-event outcomes.
Answer a few questions about your evidence
The methods at a glance
| Method | What it needs | Key assumption | EU guideline view |
|---|---|---|---|
| Naive | Single arms from different trials | None that holds: patients differ between trials | Not to be done |
| Bucher | Two trials with a common comparator | No imbalance in effect modifiers between trials | Accepted for simple networks; consistency cannot be tested |
| NMA | A connected network of trials | Consistency of direct and indirect evidence; similar effect modifiers | Accepted frequentist or Bayesian; check consistency and heterogeneity |
| Anchored MAIC | IPD for one trial, summaries for the other | All effect modifiers measured and reported; populations overlap | With care only with sufficient overlap of populations |
| Anchored STC | IPD for one trial, an outcome model | Correct outcome model including effect modifiers | With care depends on the outcome model |
| ML-NMR | IPD and summaries across a network | Correct individual-level model; covariate distributions known | With care cannot yet be applied to time-to-event data |
| Unanchored | No common comparator | Every prognostic factor and effect modifier accounted for | Highly problematic needs every prognostic factor and effect modifier |
Summarised from the HTA Coordination Group’s methodological guideline on direct and indirect comparisons (2024) and NICE DSU TSD18. Anchored comparisons should always be preferred to unanchored ones.
Three ideas to take away
Keep the randomisation
An anchored comparison only compares like with like: each medicine against the shared comparator, inside its own trial. Differences in prognosis between trials then cancel out. A naive comparison of single arms throws that protection away.
Effect modifiers are the real threat
Anchored methods survive differences in prognosis but not differences in factors that change how well a medicine works. Population-adjusted methods exist for that case, but they cost precision and depend on knowing which factors matter.
Uncertainty only grows
Variances add in every indirect step, and reweighting shrinks the effective sample. An indirect estimate is almost always less certain than the trials behind it, and an honest report says so.
Sources
- Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol 1997;50(6):683–91. doi:10.1016/s0895-4356(97)00049-8
- Signorovitch JE, Sikirica V, Erder MH, et al. Matching-adjusted indirect comparisons: a new tool for timely comparative effectiveness research. Value Health 2012;15(6):940–7. doi:10.1016/j.jval.2012.05.004
- Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NJ. NICE DSU Technical Support Document 18: methods for population-adjusted indirect comparisons in submissions to NICE. 2016. NICE DSU
- Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. 2011, updated 2016. NCBI Bookshelf
- Phillippo DM, Dias S, Ades AE, et al. Multilevel network meta-regression for population-adjusted treatment comparisons. J R Stat Soc Ser A 2020;183(3):1189–1210. doi:10.1111/rssa.12579
- Phillippo DM, Remiro-Azócar A, Heath A, et al. Effect modification and non-collapsibility together may lead to conflicting treatment decisions. Res Synth Methods 2025;16(2):323–49. doi:10.1017/rsm.2025.2
- HTA Coordination Group. Methodological guideline for quantitative evidence synthesis: direct and indirect comparisons, adopted 8 March 2024. health.ec.europa.eu
Be curious, not judgmental.
Design inspired by Spencer Gabor and Dieter Rams’s ten principles of good design.
© 2026 Jigar Rajpura. All rights reserved. No part of this site may be reproduced, distributed or adapted without written permission, except as set out in the Terms of use.
DisclaimerPersonal project, not sponsored by or affiliated with any employer (current or former) or any university I attended; views are my own. Developed with AI assistance, so errors or outdated details may exist. All medicines, trials and data are simulated. Simplified for teaching: real analyses need a statistician, a pre-specified plan and sensitivity analyses. Full disclaimer