Search

Background: Network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest but may give biased estimates if study populations differ. Population adjustment methods such as multilevel network meta-regression (ML-NMR) aim to reduce bias by adjusting for differences in study populations using individual patient data (IPD) from 1 or more studies under the conditional constancy assumption. A shared effect modifier assumption may also be necessary for identifiability. This article aims to demonstrate how the assumptions made by ML-NMR can be assessed in practice to obtain reliable treatment effect estimates in a target population., Methods: We apply ML-NMR to a network of evidence on treatments for plaque psoriasis with a mix of IPD and AgD trials reporting ordered categorical outcomes. Relative treatment effects are estimated for each trial population and for 3 external target populations represented by a registry and 2 cohort studies. We examine residual heterogeneity and inconsistency and relax the shared effect modifier assumption for each covariate in turn., Results: Estimated population-average treatment effects were similar across study populations, as differences in the distributions of effect modifiers were small. Better fit was achieved with ML-NMR than with NMA, and uncertainty was reduced by explaining within- and between-study variation. We found little evidence that the conditional constancy or shared effect modifier assumptions were invalid., Conclusions: ML-NMR extends the NMA framework and addresses issues with previous population adjustment approaches. It coherently synthesizes evidence from IPD and AgD studies in networks of any size while avoiding aggregation bias and noncollapsibility bias, allows for key assumptions to be assessed or relaxed, and can produce estimates relevant to a target population for decision-making., Highlights: Multilevel network meta-regression (ML-NMR) extends the network meta-analysis framework to synthesize evidence from networks of studies providing individual patient data or aggregate data while adjusting for differences in effect modifiers between studies (population adjustment). We apply ML-NMR to a network of treatments for plaque psoriasis with ordered categorical outcomes.We demonstrate for the first time how ML-NMR allows key assumptions to be assessed. We check for violations of conditional constancy of relative effects (such as unobserved effect modifiers) through residual heterogeneity and inconsistency and the shared effect modifier assumption by relaxing this for each covariate in turn.Crucially for decision making, population-adjusted treatment effects can be produced in any relevant target population. We produce population-average estimates for 3 external target populations, represented by the PsoBest registry and the PROSPECT and Chiricozzi 2019 cohort studies.

Guideline development requires the synthesis of evidence on several treatments of interest, typically by using network meta-analysis (NMA). Because treatment effects may be estimated imprecisely or be based on evidence lacking internal or external validity, guideline developers must assess the robustness of recommendations made on the basis of the NMA to potential limitations in the evidence. Such limitations arise because the observed estimates differ from the true effects of interest, for example, because of study biases, sampling variation, or issues of relevance. The widely used GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework aims to assess the quality of evidence supporting a recommendation by using a structured series of qualitative judgments. This article argues that GRADE approaches proposed for NMA are insufficient for the purposes of guideline development, because the influence of the evidence on the final recommendation is not taken into account. It outlines threshold analysis as an alternative approach, demonstrating the method with 2 examples of clinical guidelines from the National Institute for Health and Care Excellence (NICE) in the United Kingdom. Threshold analysis quantifies precisely how much the evidence could change (for any reason, such as potential biases, or simply sampling variation) before the recommendation changes, and what the revised recommendation would be. If it is judged that the evidence could not plausibly change by more than this amount, then the recommendation is considered robust; otherwise, it is sensitive to plausible changes in the evidence. In this manner, threshold analysis directly informs decision makers and guideline developers of the robustness of treatment recommendations.

Objective: To assess the reliability of treatment recommendations based on network meta-analysis (NMA)., Study Design and Setting: We consider evidence in an NMA to be potentially biased. Taking each pairwise contrast in turn, we use a structured series of threshold analyses to ask: (1) "How large would the bias in this evidence base have to be before it changed our decision?" and (2) "If the decision changed, what is the new recommendation?" We illustrate the method via two NMAs in which a Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessment for NMAs has been implemented: weight loss and osteoporosis., Results: Four of the weight-loss NMA estimates were assessed as "low" and six as "moderate" quality by GRADE; for osteoporosis, six were "low," nine were "moderate," and 1 was "high." The threshold analysis suggests plausible bias in 3 of 10 estimates in the weight-loss network could have changed the treatment recommendation. For osteoporosis, plausible bias in 6 of 16 estimates could change the recommendation. There was no relation between plausible bias changing a treatment recommendation and the original GRADE assessments., Conclusions: Reliability judgments on individual NMA contrasts do not help decision makers understand whether a treatment recommendation is reliable. Threshold analysis reveals whether the final recommendation is robust against plausible degrees of bias in the data., (Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.)

Network meta-analysis enables the simultaneous synthesis of a network of clinical trials comparing any number of treatments. Potential inconsistencies between estimates of relative treatment effects are an important concern, and several methods to detect inconsistency have been proposed. This paper is concerned with the node-splitting approach, which is particularly attractive because of its straightforward interpretation, contrasting estimates from both direct and indirect evidence. However, node-splitting analyses are labour-intensive because each comparison of interest requires a separate model. It would be advantageous if node-splitting models could be estimated automatically for all comparisons of interest. We present an unambiguous decision rule to choose which comparisons to split, and prove that it selects only comparisons in potentially inconsistent loops in the network, and that all potentially inconsistent loops in the network are investigated. Moreover, the decision rule circumvents problems with the parameterisation of multi-arm trials, ensuring that model generation is trivial in all cases. Thus, our methods eliminate most of the manual work involved in using the node-splitting approach, enabling the analyst to focus on interpreting the results., (© 2015 The Authors Research Synthesis Methods Published by John Wiley & Sons Ltd.)

We respond to discussant comments on our paper “Twenty years of network meta‐analysis: Continuing controversies and recent developments” (https://doi.org/10.1002/jrsm.1700) and raise some additional points for consideration, including: the way in which methodological guidance is generated; integration of the estimand framework with evidence synthesis; and implications of the European Joint Clinical Assessment. We ask: what properties are required of population adjustment methods to enable transparent and consistent decision‐making? We also ask why individual patient data is not routinely made available to re‐imbursement authorities and clinical guideline developers. [ABSTRACT FROM AUTHOR]

Network meta‐analysis (NMA) is an extension of pairwise meta‐analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re‐imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta‐analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta‐analyses in clinical medicine on average with less than six studies. This has been followed by narrowly‐focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well‐established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi‐level network meta‐regression. [ABSTRACT FROM AUTHOR]

Randomised controlled trials of cancer treatments typically report progression free survival (PFS) and overall survival (OS) outcomes. Existing methods to synthesise evidence on PFS and OS either rely on the proportional hazards assumption or make parametric assumptions which may not capture the diverse survival curve shapes across studies and treatments. Furthermore, PFS and OS are not independent; OS is the sum of PFS and post‐progression survival (PPS). Our aim was to develop a non‐parametric approach for jointly synthesising evidence from published Kaplan–Meier survival curves of PFS and OS without assuming proportional hazards. Restricted mean survival times (RMST) are estimated by the area under the survival curves (AUCs) up to a restricted follow‐up time. The correlation between AUCs due to the constraint that OS > PFS is estimated using bootstrap re‐sampling. Network meta‐analysis models are given for RMST for PFS and PPS and ensure that OS = PFS + PPS. Both additive and multiplicative network meta‐analysis models are presented to obtain relative treatment effects as either differences or ratios of RMST. The methods are illustrated with a network meta‐analysis of treatments for stage IIIA‐N2 non‐small cell lung cancer. The approach has implications for health economic models of cancer treatments, which require estimates of the mean time spent in the PFS and PPS health‐states. The methods can be applied to a single time‐to‐event outcome, and so have wide applicability in any field where time‐to‐event outcomes are reported, the proportional hazards assumption is in doubt, and survival curve shapes differ across studies and interventions. [ABSTRACT FROM AUTHOR]

Summary: Standard network meta‐analysis (NMA) and indirect comparisons combine aggregate data from multiple studies on treatments of interest, assuming that any effect modifiers are balanced across populations. Population adjustment methods relax this assumption using individual patient data from one or more studies. However, current matching‐adjusted indirect comparison and simulated treatment comparison methods are limited to pairwise indirect comparisons and cannot predict into a specified target population. Existing meta‐regression approaches incur aggregation bias. We propose a new method extending the standard NMA framework. An individual level regression model is defined, and aggregate data are fitted by integrating over the covariate distribution to form the likelihood. Motivated by the complexity of the closed form integration, we propose a general numerical approach using quasi‐Monte‐Carlo integration. Covariate correlation structures are accounted for by using copulas. Crucially for decision making, comparisons may be provided in any target population with a given covariate distribution. We illustrate the method with a network of plaque psoriasis treatments. Estimated population‐average treatment effects are similar across study populations, as differences in the distributions of effect modifiers are small. A better fit is achieved than a random effects NMA, uncertainty is substantially reduced by explaining within‐ and between‐study variation, and estimates are more interpretable. [ABSTRACT FROM AUTHOR]

Objectives: Indirect comparisons via a common comparator (anchored comparisons) are commonly used in health technology assessment. However, common comparators may not be available, or the comparison may be biased due to differences in effect modifiers between the included studies. Recently proposed population adjustment methods aim to adjust for differences between study populations in the situation where individual patient data are available from at least one study, but not all studies. They can also be used when there is no common comparator or for single-arm studies (unanchored comparisons). We aim to characterise the use of population adjustment methods in technology appraisals (TAs) submitted to the United Kingdom National Institute for Health and Care Excellence (NICE).Methods: We reviewed NICE TAs published between 01/01/2010 and 20/04/2018.Results: Population adjustment methods were used in 7 percent (18/268) of TAs. Most applications used unanchored comparisons (89 percent, 16/18), and were in oncology (83 percent, 15/18). Methods used included matching-adjusted indirect comparisons (89 percent, 16/18) and simulated treatment comparisons (17 percent, 3/18). Covariates were included based on: availability, expert opinion, effective sample size, statistical significance, or cross-validation. Larger treatment networks were commonplace (56 percent, 10/18), but current methods cannot account for this. Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk.Conclusions: Population adjustment methods are becoming increasingly common in NICE TAs, although their impact on decisions has been limited to date. Further research is needed to improve upon current methods, and to investigate their properties in simulation studies. [ABSTRACT FROM AUTHOR]

Network meta-analysis enables the simultaneous synthesis of a network of clinical trials comparing any number of treatments. Potential inconsistencies between estimates of relative treatment effects are an important concern, and several methods to detect inconsistency have been proposed. This paper is concerned with the node-splitting approach, which is particularly attractive because of its straightforward interpretation, contrasting estimates from both direct and indirect evidence. However, node-splitting analyses are labour-intensive because each comparison of interest requires a separate model. It would be advantageous if node-splitting models could be estimated automatically for all comparisons of interest. We present an unambiguous decision rule to choose which comparisons to split, and prove that it selects only comparisons in potentially inconsistent loops in the network, and that all potentially inconsistent loops in the network are investigated. Moreover, the decision rule circumvents problems with the parameterisation of multi-arm trials, ensuring that model generation is trivial in all cases. Thus, our methods eliminate most of the manual work involved in using the node-splitting approach, enabling the analyst to focus on interpreting the results. © 2015 The Authors Research Synthesis Methods Published by John Wiley & Sons Ltd. [ABSTRACT FROM AUTHOR]

Advanced evidence synthesis techniques such as indirect or mixed treatment comparisons provide powerful analytic tools to inform decision making. In some cases, however, existing research is limited in quantity and/or existing research data are 'sparse'. We demonstrate how modelling assumptions in evidence synthesis can be explored in the face of limited and sparse data by using an example where estimates of relative treatment effects were required in a synthesis of the available evidence regarding treatments for grade 3 or 4 pressure ulcers. [ABSTRACT FROM AUTHOR]

Meta-analyses that simultaneously compare multiple treatments (usually referred to as network meta-analyses or mixed treatment comparisons) are becoming increasingly common. An important component of a network meta-analysis is an assessment of the extent to which different sources of evidence are compatible, both substantively and statistically. A simple indirect comparison may be confounded if the studies involving one of the treatments of interest are fundamentally different from the studies involving the other treatment of interest. Here, we discuss methods for addressing inconsistency of evidence from comparative studies of different treatments. We define and review basic concepts of heterogeneity and inconsistency, and attempt to introduce a distinction between 'loop inconsistency' and 'design inconsistency'. We then propose that the notion of design-by-treatment interaction provides a useful general framework for investigating inconsistency. In particular, using design-by-treatment interactions successfully addresses complications that arise from the presence of multi-arm trials in an evidence network. We show how the inconsistency model proposed by Lu and Ades is a restricted version of our full design-by-treatment interaction model and that there may be several distinct Lu-Ades models for any particular data set. We introduce novel graphical methods for depicting networks of evidence, clearly depicting multi-arm trials and illustrating where there is potential for inconsistency to arise. We apply various inconsistency models to data from trials of different comparisons among four smoking cessation interventions and show that models seeking to address loop inconsistency alone can run into problems. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Multi-arm trials (trials with more than two arms) are particularly valuable forms of evidence for network meta-analysis (NMA). Trial results are available either as arm-level summaries, where effect measures are reported for each arm, or as contrast-level summaries, where the differences in effect between arms compare with the control arm chosen for the trial. We show that likelihood-based inference in both contrast-level and arm-level formats is identical if there are only two-arm trials, but that if there are multi-arm trials, results from the contrast-level format will be incorrect unless correlations are accounted for in the likelihood. We review Bayesian and frequentist software for NMA with multi-arm trials that can account for this correlation and give an illustrative example of the difference in estimates that can be introduced if the correlations are not incorporated. We discuss methods of imputing correlations when they cannot be derived from the reported results and urge trialists to report the standard error for the control arm even if only contrast-level summaries are reported. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

There is good empirical evidence that specific flaws in the conduct of randomized controlled trials are associated with exaggeration of treatment effect estimates. Mixed treatment comparison meta-analysis, which combines data from trials on several treatments that form a network of comparisons, has the potential both to estimate bias parameters within the synthesis and to produce bias-adjusted estimates of treatment effects. We present a hierarchical model for bias with common mean across treatment comparisons of active treatment versus control. It is often unclear, from the information that is reported, whether a study is at risk of bias or not. We extend our model to estimate the probability that a particular study is biased, where the probabilities for the ‘unclear’ studies are drawn from a common beta distribution. We illustrate these methods with a synthesis of 130 trials on four fluoride treatments and two control interventions for the prevention of dental caries in children. Whether there is adequate allocation concealment and/or blinding are considered as indicators of whether a study is at risk of bias. Bias adjustment reduces the estimated relative efficacy of the treatments and the extent of between-trial heterogeneity. [ABSTRACT FROM AUTHOR]

Evidence for the efficacy of biologic therapies in inflammatory arthritis comes overwhelmingly from placebo-controlled trials. Increasingly, however, authorities responsible for purchasing and re-imbursement have tried to determine whether there are differences between these powerful new therapies, which would lead them to recommend some in preference to others, either on grounds of efficacy or cost-effectiveness. In the absence of head-to-head trial comparisons, indirect comparisons may be used. Furthermore, network meta-analysis, also known as mixed treatment comparisons can combine information from trials in a connected network. These methods allow inferences about head-to-head comparisons even when there is little or no head-to-head evidence, which has caused some concern. In this article we briefly review these methodologies and describe results from recent applications to inflammatory arthritis in the clinical literature. We then focus on how the methodologies are used in decision making, taking as an illustration some recent technology appraisals conducted by the National Institute for Health and Clinical Excellence in the UK. We conclude that, in practice, the key decisions have been based on results from placebo-controlled trials. [ABSTRACT FROM PUBLISHER]