Search

Abstract Background Transitivity assumption is the cornerstone of network meta-analysis (NMA). Investigating the plausibility of transitivity can unveil the credibility of NMA results. The commonness of transitivity was examined based on study dissimilarities regarding several study-level aggregate clinical and methodological characteristics reported in the systematic reviews. The present study also demonstrated the disadvantages of using multiple statistical tests to assess transitivity and compared the conclusions drawn from multiple statistical tests with those from the approach of study dissimilarities for transitivity assessment. Methods An empirical study was conducted using 209 published systematic reviews with NMA to create a database of study-level aggregate clinical and methodological characteristics found in the tracenma R package. For each systematic review, the network of the primary outcome was considered to create a dataset with extracted study-level aggregate clinical and methodological characteristics reported in the systematic review that may act as effect modifiers. Transitivity was evaluated by calculating study dissimilarities based on the extracted characteristics to provide a measure of overall dissimilarity within and between the observed treatment comparisons. Empirically driven thresholds of low dissimilarity were employed to determine the proportion of datasets with evidence of likely intransitivity. One-way ANOVA and chi-squared test were employed for each characteristic to investigate comparison dissimilarity at a significance level of 5%. Results Study dissimilarities covered a wide range of possible values across the datasets. A ‘likely concerning’ extent of study dissimilarities, both intra-comparison and inter-comparison, dominated the analysed datasets. Using a higher dissimilarity threshold, a ‘likely concerning’ extent of study dissimilarities persisted for objective outcomes but decreased substantially for subjective outcomes. A likely intransitivity prevailed in all datasets; however, using a higher dissimilarity threshold resulted in few networks with transitivity for semi-objective and subjective outcomes. Statistical tests were feasible in 127 (61%) datasets, yielding conflicting conclusions with the approach of study dissimilarities in many datasets. Conclusions Study dissimilarity, manifested from variations in the effect modifiers’ distribution across the studies, should be expected and properly quantified. Measuring the overall study dissimilarity between observed comparisons and comparing it with a proper threshold can aid in determining whether concerns of likely intransitivity are warranted.

Abstract Background The standard approach to local inconsistency assessment typically relies on testing the conflict between the direct and indirect evidence in selected treatment comparisons. However, statistical tests for inconsistency have low power and are subject to misinterpreting a p-value above the significance threshold as evidence of consistency. Methods We propose a simple framework to interpret local inconsistency based on the average Kullback–Leibler divergence (KLD) from approximating the direct with the corresponding indirect estimate and vice versa. Our framework uses directly the mean and standard error (or posterior mean and standard deviation) of the direct and indirect estimates obtained from a local inconsistency method to calculate the average KLD measure for selected comparisons. The average KLD values are compared with a semi-objective threshold to judge the inconsistency as acceptably low or material. We exemplify our novel interpretation approach using three networks with multiple treatments and multi-arm studies. Results Almost all selected comparisons in the networks were not associated with statistically significant inconsistency at a significance level of 5%. The proposed interpretation framework indicated 14%, 66%, and 75% of the selected comparisons with an acceptably low inconsistency in the corresponding networks. Overall, information loss was more notable when approximating the posterior density of the indirect estimates with that of the direct estimates, attributed to indirect estimates being more imprecise. Conclusions Using the concept of information loss between two distributions alongside a semi-objectively defined threshold helped distinguish target comparisons with acceptably low inconsistency from those with material inconsistency when statistical tests for inconsistency were inconclusive.

Abstract Background The transitivity assumption is the cornerstone of network meta-analysis (NMA). Violating transitivity compromises the credibility of the indirect estimates and, by extent, the estimated treatment effects of the comparisons in the network. The present study offers comprehensive empirical evidence on the completeness of reporting and evaluating transitivity in systematic reviews with multiple interventions. Methods We screened the datasets of two previous empirical studies, resulting in 361 systematic reviews with NMA published between January 2011 and April 2015. We updated our evidence base with an additional 360 systematic reviews with NMA published between 2016 and 2021, employing a pragmatic approach. We devised assessment criteria for reporting and evaluating transitivity using relevant methodological literature and compared their reporting frequency before and after the PRISMA-NMA statement. Results Systematic reviews published after PRISMA-NMA were more likely to provide a protocol (odds ratio (OR): 3.94, 95% CI: 2.79–5.64), pre-plan the transitivity evaluation (OR: 3.01, 95% CI: 1.54–6.23), and report the evaluation and results (OR: 2.10, 95% CI: 1.55–2.86) than those before PRISMA-NMA. However, systematic reviews after PRISMA-NMA were less likely to define transitivity (OR: 0.57, 95% CI: 0.42–0.79) and discuss the implications of transitivity (OR: 0.48, 95% CI: 0.27–0.85) than those published before PRISMA-NMA. Most systematic reviews evaluated transitivity statistically than conceptually (40% versus 12% before PRISMA-NMA, and 54% versus 11% after PRISMA-NMA), with consistency evaluation being the most preferred (34% before versus 47% after PRISMA-NMA). One in five reviews inferred the plausibility of the transitivity (22% before versus 18% after PRISMA-NMA), followed by 11% of reviews that found it difficult to judge transitivity due to insufficient data. In justifying their conclusions, reviews considered mostly the comparability of the trials (24% before versus 30% after PRISMA-NMA), followed by the consistency evaluation (23% before versus 16% after PRISMA-NMA). Conclusions Overall, there has been a slight improvement in reporting and evaluating transitivity since releasing PRISMA-NMA, particularly in items related to the systematic review report. Nevertheless, there has been limited attention to pre-planning the transitivity evaluation and low awareness of the conceptual evaluation methods that align with the nature of the assumption.

Abstract Background To investigate the prevalence of robust conclusions in systematic reviews addressing missing (participant) outcome data via a novel framework of sensitivity analyses and examine the agreement with the current sensitivity analysis standards. Methods We performed an empirical study on systematic reviews with two or more interventions. Pairwise meta-analyses (PMA) and network meta-analyses (NMA) were identified from empirical studies on the reporting and handling of missing outcome data in systematic reviews. PMAs with at least three studies and NMAs with at least three interventions on one primary outcome were considered eligible. We applied Bayesian methods to obtain the summary effect estimates whilst modelling missing outcome data under the missing-at-random assumption and different assumptions about the missingness mechanism in the compared interventions. The odds ratio in the logarithmic scale was considered for the binary outcomes and the standardised mean difference for the continuous outcomes. We calculated the proportion of primary analyses with robust and frail conclusions, quantified by our proposed metric, the robustness index (RI), and current sensitivity analysis standards. Cohen’s kappa statistic was used to measure the agreement between the conclusions derived by the RI and the current sensitivity analysis standards. Results One hundred eight PMAs and 34 NMAs were considered. When studies with a substantial number of missing outcome data dominated the analyses, the number of frail conclusions increased. The RI indicated that 59% of the analyses failed to demonstrate robustness compared to 39% when the current sensitivity analysis standards were employed. Comparing the RI with the current sensitivity analysis standards revealed that two in five analyses yielded contradictory conclusions concerning the robustness of the primary analysis results. Conclusions Compared with the current sensitivity analysis standards, the RI offers an explicit definition of similar results and does not unduly rely on statistical significance. Hence, it may safeguard against possible spurious conclusions regarding the robustness of the primary analysis results.

Abstract Objectives To systematically assess the long‐term outcome (≥5 years) of root coverage procedures reported in controlled clinical trials. Material and Methods Literature search was performed according to the PRISMA guidelines with the following eligibility criteria: (a) English or German language; (b) controlled (CT) or randomised controlled clinical trials (RCT); (c) root coverage procedure with ≥5 years follow‐up; and (d) clinical treatment effect size and/or patient‐related outcome measures (PROMs) reported. Results Four CT and 14 RCT with a follow‐up of 5–20 years fulfilled the eligibility criteria; sample size per study ranged from 8 to 70 patients contributing with 18–149 sites. Coronally advanced flap (CAF) and CAF + connective tissue graft (CTG) were the prevalent treatments (i.e., in 24 and 38% of the groups, respectively), while other flap designs and adjuncts (i.e., enamel matrix derivative, bone graft, collagen membrane) were represented only once. For single Miller class I/II gingival recessions (GR), CAF + CTG appeared advantageous compared to other techniques, and provided low residual recession depths (i.e., ≤0.5 mm), and complete root coverage in ≥2/3 of the patients; similar tendency was observed for multiple GR. No data on Miller class III/IV GR is available. No meta‐analysis was feasible due to lack of similarity in the clinical and methodological characteristics across the trials and observed comparisons of interventions. Conclusions CAF + CTG appears to be the ‘gold standard’ technique for the treatment of single and multiple Miller class I/II GR also in regard to long‐term (i.e., ≥5 years of follow‐up) treatment outcomes. There is little information regarding the performance, on the long‐term, of other techniques and adjuncts.

Acute myeloid leukemia (AML) patients with minimal residual disease and receiving allogeneic hematopoietic stem cell transplantation (HCT) have poor survival. Adoptive administration of dendritic cells (DCs) presenting the Wilms tumor protein 1 (WT1) leukemia-associated antigen can potentially stimulate de novo T and B cell development to harness the graft-versus-leukemia (GvL) effect after HCT. We established a simple and fast genetic modification of monocytes for simultaneous lentiviral expression of a truncated WT1 antigen (tWT1), granulocyte macrophage-colony-stimulating factor (GM-CSF), and interferon (IFN)-α, promoting their self-differentiation into potent “induced DCs” (iDCtWT1). A tricistronic integrase-defective lentiviral vector produced under good manufacturing practice (GMP)-like conditions was validated. Transduction of CD14+ monocytes isolated from peripheral blood, cord blood, and leukapheresis material effectively induced their self-differentiation. CD34+ cell-transplanted Nod.Rag.Gamma (NRG)- and Nod.Scid.Gamma (NSG) mice expressing human leukocyte antigen (HLA)-A∗0201 (NSG-A2)-immunodeficient mice were immunized with autologous iDCtWT1. Both humanized mouse models showed improved development and maturation of human T and B cells in the absence of adverse effects. Toward clinical use, manufacturing of iDCtWT1 was up scaled and streamlined using the automated CliniMACS Prodigy system. Proof-of-concept clinical-scale runs were feasible, and the 38-h process enabled standardized production and high recovery of a cryopreserved cell product with the expected identity characteristics. These results advocate for clinical trials testing iDCtWT1 to boost GvL and eradicate leukemia.

Abstract Background Trials with binary outcomes can be synthesised using within-trial exact likelihood or approximate normal likelihood in one-stage or two-stage approaches, respectively. The performance of the one-stage and the two-stage approaches has been documented extensively in the literature. However, little is known about how these approaches behave in the presence of missing outcome data (MOD), which are ubiquitous in clinical trials. In this work, we compare the one-stage versus two-stage approach via a pattern-mixture model in the network meta-analysis using Bayesian methods to handle MOD appropriately. Methods We used 29 published networks to empirically compare the two approaches concerning the relative treatment effects of several competing interventions and the between-trial variance (τ 2), while considering the extent and level of balance of MOD in the included trials. We additionally conducted a simulation study to compare the competing approaches regarding the bias and width of the 95% credible interval of the (summary) log odds ratios (OR) and τ 2 in the presence of moderate and large MOD. Results The empirical study did not reveal any systematic bias between the compared approaches regarding the log OR, but showed systematically larger uncertainty around the log OR under the one-stage approach for networks with at least one small trial or low event risk and moderate MOD. For these networks, the simulation study revealed that the bias in log OR for comparisons with the reference intervention in the network was relatively higher in the two-stage approach. Contrariwise, the bias in log OR for the remaining comparisons was relatively higher in the one-stage approach. Overall, bias increased for large MOD. For these networks, the empirical results revealed slightly higher τ 2 estimates under the one-stage approach irrespective of the extent of MOD. The one-stage approach also led to less precise log OR and τ 2 when compared with the two-stage approach for large MOD. Conclusions Due to considerable bias in the log ORs overall, especially for large MOD, none of the competing approaches was superior. Until a more competent model is developed, the researchers may prefer the one-stage approach to handle MOD, while acknowledging its limitations.

Abstract Background Missing participant outcome data (MOD) are ubiquitous in systematic reviews with network meta-analysis (NMA) as they invade from the inclusion of clinical trials with reported participant losses. There are available strategies to address aggregate MOD, and in particular binary MOD, while considering the missing at random (MAR) assumption as a starting point. Little is known about their performance though regarding the meta-analytic parameters of a random-effects model for aggregate binary outcome data as obtained from trial-reports (i.e. the number of events and number of MOD out of the total randomised per arm). Methods We used four strategies to handle binary MOD under MAR and we classified these strategies to those modelling versus excluding/imputing MOD and to those accounting for versus ignoring uncertainty about MAR. We investigated the performance of these strategies in terms of core NMA estimates by performing both an empirical and simulation study using random-effects NMA based on electrical network theory. We used Bland-Altman plots to illustrate the agreement between the compared strategies, and we considered the mean bias, coverage probability and width of the confidence interval to be the frequentist measures of performance. Results Modelling MOD under MAR agreed with exclusion and imputation under MAR in terms of estimated log odds ratios and inconsistency factor, whereas accountability or not of the uncertainty regarding MOD affected intervention hierarchy and precision around the NMA estimates: strategies that ignore uncertainty about MOD led to more precise NMA estimates, and increased between-trial variance. All strategies showed good performance for low MOD ( 20%), inconsistent evidence and substantial between-trial variance, especially for strategies that ignore uncertainty due to MOD. Conclusions The analysts should avoid applying strategies that manipulate MOD before analysis (i.e. exclusion and imputation) as they implicate the inferences negatively. Modelling MOD, on the other hand, via a pattern-mixture model to propagate the uncertainty about MAR assumption constitutes both conceptually and statistically proper strategy to address MOD in a systematic review.

Abstract Background A number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews. However, none of these have been evaluated empirically in a series of published systematic reviews. Methods Using published systematic reviews with network meta-analysis (NMA) from a wide range of health-related fields, we evaluated comparatively the most frequently described Bayesian modelling strategies for MOD in terms of log odds ratio (log OR), between-trial variance, inconsistency factor (i.e. difference between direct and indirect estimates for a comparison), surface under the cumulative ranking (SUCRA) and rankings. We extended the Bayesian random-effects NMA model to incorporate the informative missingness odds ratio (IMOR) parameter, and applied the node-splitting approach to investigate inconsistency locally. We considered both pattern-mixture and selection models, different structures for prior distribution of log IMOR, and different scenarios for MOD. To illustrate level of agreement between different strategies and scenarios, we used Bland-Altman plots. Results Addressing MOD using extreme scenarios and ignoring the uncertainty about the scenarios led to systematically different and more precise log ORs compared to modelling MOD under the missing at random (MAR) assumption. Hierarchical structure of log IMORs led to lower between-trial variance, especially in the case of substantial MOD. Assuming common-within-network or trial-specific log IMORs yielded similar posterior results for all NMA estimates, whereas intervention-specific structure systematically inflated uncertainty around log ORs and SUCRAs. Pattern-mixture model agreed with selection model, particularly under the trial-specific structure; however, selection model systematically reduced precision around log IMORs. Overall, different strategies and scenarios mostly had good agreement in the case of low MOD. Conclusions Addressing MOD using extreme scenarios and/or ignoring the uncertainty about the scenarios may negatively affect NMA estimates. Modelling MOD via the IMOR parameter can ensure bias-adjusted estimates and offer valuable insights into missingness mechanisms. The researcher should seek an expert opinion in order to decide on the structure of log IMOR that best aligns to the condition and interventions studied and to define a proper prior distribution for log IMOR. Our findings also apply to pairwise meta-analyses.

Abstract Background To provide empirical evidence about prevalence, reporting and handling of missing outcome data in systematic reviews with network meta-analysis and acknowledgement of their impact on the conclusions. Methods We conducted a systematic survey including all published systematic reviews of randomized controlled trials comparing at least three interventions from January 1, 2009 until March 31, 2017. Results We retrieved 387 systematic reviews with network meta-analysis. Description of missing outcome data was available in 63 reviews. Intention-to-treat analysis was the most prevalent method (71%), followed by missing outcome data investigated as secondary outcome (e.g., acceptability) (40%). Bias due to missing outcome data was evaluated in half the reviews with explicit judgments in 18 (10%) reviews. Only 88 reviews interpreted their results acknowledging the implications of missing outcome data and mostly using the network meta-analysis results on missing outcome data as secondary outcome. We were unable to judge the actual strategy applied to deal with missing outcome data in 65% of the reviews due to insufficient information. Six percent of network meta-analyses were re-analyzed in sensitivity analysis considering missing outcome data, while 4% explicitly justified the strategy for dealing with missing outcome data. Conclusions The description and handling of missing outcome data as well as the acknowledgment of their implications for the conclusions from network meta-analysis are deemed underreported.

Abstract Background A number of papers have proposed or evaluated the delayed-start design as an alternative to the standard two-arm parallel group randomized clinical trial (RCT) design in the field of rare disease. However the discussion is felt to lack a sufficient degree of consideration devoted to the true virtues of the delayed start design and the implications either in terms of required sample-size, overall information, or interpretation of the estimate in the context of small populations. Objectives To evaluate whether there are real advantages of the delayed-start design particularly in terms of overall efficacy and sample size requirements as a proposed alternative to the standard parallel group RCT in the field of rare disease. Methods We used a real-life example to compare the delayed-start design with the standard RCT in terms of sample size requirements. Then, based on three scenarios regarding the development of the treatment effect over time, the advantages, limitations and potential costs of the delayed-start design are discussed. Results We clarify that delayed-start design is not suitable for drugs that establish an immediate treatment effect, but for drugs with effects developing over time, instead. In addition, the sample size will always increase as an implication for a reduced time on placebo resulting in a decreased treatment effect. Conclusions A number of papers have repeated well-known arguments to justify the delayed-start design as appropriate alternative to the standard parallel group RCT in the field of rare disease and do not discuss the specific needs of research methodology in this field. The main point is that a limited time on placebo will result in an underestimated treatment effect and, in consequence, in larger sample size requirements compared to those expected under a standard parallel-group design. This also impacts on benefit-risk assessment.

Human cytomegalovirus (HCMV) latency is typically harmless but reactivation can be largely detrimental to immune compromised hosts. We modeled latency and reactivation using a traceable HCMV laboratory strain expressing the Gaussia luciferase reporter gene (HCMV/GLuc) in order to interrogate the viral modulatory effects on the human adaptive immunity. Humanized mice with long-term (more than 17 weeks) steady human T and B cell immune reconstitutions were infected with HCMV/GLuc and 7 weeks later were further treated with granulocyte-colony stimulating factor (G-CSF) to induce viral reactivations. Whole body bio-luminescence imaging analyses clearly differentiated mice with latent viral infections vs. reactivations. Foci of vigorous viral reactivations were detectable in liver, lymph nodes and salivary glands. The number of viral genome copies in various tissues increased upon reactivations and were detectable in sorted human CD14+, CD169+, and CD34+ cells. Compared with non-infected controls, mice after infections and reactivations showed higher thymopoiesis, systemic expansion of Th, CTL, Treg, and Tfh cells and functional antiviral T cell responses. Latent infections promoted vast development of memory CD4+ T cells while reactivations triggered a shift toward effector T cells expressing PD-1. Further, reactivations prompted a marked development of B cells, maturation of IgG+ plasma cells, and HCMV-specific antibody responses. Multivariate statistical methods were employed using T and B cell immune phenotypic profiles obtained with cells from several tissues of individual mice. The data was used to identify combinations of markers that could predict an HCMV infection vs. reactivation status. In spleen, but not in lymph nodes, higher frequencies of effector CD4+ T cells expressing PD-1 were among the factors most suited to distinguish HCMV reactivations from infections. These results suggest a shift from a T cell dominated immune response during latent infections toward an exhausted T cell phenotype and active humoral immune response upon reactivations. In sum, this novel in vivo humanized model combined with advanced analyses highlights a dynamic system clearly specifying the immunological spatial signatures of HCMV latency and reactivations. These signatures can be merged as predictive biomarker clusters that can be applied in the clinical translation of new therapies for the control of HCMV reactivation.

Mice transplanted with human cord blood-derived hematopoietic stem cells (HSCs) became a powerful experimental tool for studying the heterogeneity of human immune reconstitution and immune responses in vivo. Yet, analyses of human T cell maturation in humanized models have been hampered by an overall low immune reactivity and lack of methods to define predictive markers of responsiveness. Long-lived human lentiviral induced dendritic cells expressing the cytomegalovirus pp65 protein (iDCpp65) promoted the development of pp65-specific human CD8+ T cell responses in NOD.Cg-Rag1tm1Mom-Il2rγtm1Wj humanized mice through the presentation of immune-dominant antigenic epitopes (signal 1), expression of co-stimulatory molecules (signal 2), and inflammatory cytokines (signal 3). We exploited this validated system to evaluate the effects of mouse sex in the dynamics of T cell homing and maturation status in thymus, blood, bone marrow, spleen, and lymph nodes. Statistical analyses of cell relative frequencies and absolute numbers demonstrated higher CD8+ memory T cell reactivity in spleen and lymph nodes of immunized female mice. In order to understand to which extent the multidimensional relation between organ-specific markers predicted the immunization status, the immunophenotypic profiles of individual mice were used to train an artificial neural network designed to discriminate immunized and non-immunized mice. The highest accuracy of immune reactivity prediction could be obtained from lymph node markers of female mice (77.3%). Principal component analyses further identified clusters of markers best suited to describe the heterogeneity of immunization responses in vivo. A correlation analysis of these markers reflected a tissue-specific impact of immunization. This allowed for an organ-resolved characterization of the immunization status of individual mice based on the identified set of markers. This new modality of multidimensional analyses can be used as a framework for defining minimal but predictive signatures of human immune responses in mice and suggests critical markers to characterize responses to immunization after HSC transplantation.