Search

Objectives To evaluate whether noninferiority trials are designed to adequately preserve the historical treatment effect of their active comparators. Study Design and Setting We reviewed 162 noninferiority trials published in high-impact medical journals. We assessed whether trials were designed to ensure that interventions could only be declared noninferior if they preserved at least 50% of the active comparator's historical treatment effect. Results Only 25 of 162 trials (15%) were designed so that interventions could only be declared noninferior if they preserved at least 50% of the active comparator's historical treatment effect. Most trials did not provide evidence that the active comparator was effective (n = 101), provided inadequate evidence (n = 18), or used a noninferiority margin that was too wide (n = 18). In a subset of 61 noninferiority trials which referenced a prior randomized trial or meta-analysis evaluating the active comparator, only 25 (41%) used a noninferiority margin small enough to preserve at least 50% of the active comparator's treatment effect. Overall, 14 of 162 noninferiority trials (9%) would have allowed the intervention to be declared noninferior even if it was worse than either placebo or another historical control. Conclusion Most noninferiority trials published in major medical journals could allow erroneous declarations of noninferiority.

Objectives Results from randomized trials can depend on the statistical analysis approach used. It is important to prespecify the analysis approach in the trial protocol to avoid selective reporting of analyses based on those which provide the most favourable results. We undertook a review of published trial protocols to assess how often the statistical analysis of the primary outcome was adequately prespecified. Methods We searched protocols of randomized trials indexed in PubMed in November 2016. We identified whether the following aspects of the statistical analysis approach for the primary outcome were adequately prespecified: (1) analysis population; (2) analysis model; (3) use of covariates; and (4) method of handling missing data. Results We identified 99 eligible protocols. Very few protocols adequately prespecified the analysis population (8/99, 8%), analysis model (27/99, 27%), covariates (40/99, 40%), or approach to handling missing data (10/99, 10%). Most protocols did not adequately predefine any of these four aspects of their statistical analysis approach (39%) or predefined only one aspect (36%). No protocols adequately predefined all four aspects of the analysis. Conclusion The statistical analysis approach is rarely prespecified in published trial protocols. This may allow selective reporting of results based on different analyses.

Objective Re-randomization trials allow patients to be re-enrolled for multiple treatment episodes. However, it remains uncertain to what extent re-randomization improves recruitment compared to parallel group designs or whether treatment estimates might be affected. Study Design and Setting We evaluated trials included in a recent Cochrane review of granulocyte colony-stimulating factors for patients with febrile neutropenia. We assessed the recruitment benefits of re-randomization trials; compared treatment effect estimates between re-randomization and parallel group designs; and assessed whether re-randomization led to higher rates of non-compliance and loss to follow-up in subsequent episodes. Results We included 14 trials (5 re-randomization and 9 parallel group). The re-randomization trials recruited a median of 25% (range 16–66%) more episodes on average than they would have under a parallel-group design. Treatment effect estimates were similar between re-randomization and parallel group trials across all outcomes, though confidence intervals were wide. The re-randomization trials in this review reported no loss to follow-up and low rates of non-compliance (median 1.7%, range 0–8.9%). Conclusions In the setting of febrile neutropenia, re-randomization increased recruitment while providing similar estimates of treatment effect to parallel group trials, with minimal loss to follow-up or non-compliance. It appears to be safe and efficient alternative to parallel group designs in this setting.

Objectives Factorial designs can allow efficient evaluation of multiple treatments within a single trial. We evaluated the design, analysis, and reporting in a sample of factorial trials. Study Design and Setting Review of 2 × 2 factorial trials evaluating health-related interventions and outcomes in humans. Using Medline, we identified articles published between January 2015 and March 2018. We randomly selected 100 articles for inclusion. Results Most trials (78%) did not provide a rationale for using a factorial design. Only 63 trials (63%) assessed the interaction for the primary outcome, and 39/63 (62%) made a further assessment for at least one secondary outcome. 12/63 trials (19%) identified a significant interaction for the primary outcome and 16/39 trials (41%) for at least one secondary outcome. Inappropriate methods of analysis to protect against potential negative effects from interactions were common, with 18 trials (18%) choosing the analysis method based on a preliminary test for interaction, and 13% (n = 10/75) of those conducting a factorial analysis including an interaction term in the model. Conclusion Reporting of factorial trials was often suboptimal, and assessment of interactions was poor. Investigators often used inappropriate methods of analysis to try to protect against adverse effects of interactions.

Objective To assess the design and statistical methods used in cluster-randomized crossover (CRXO) trials. Study Design and Setting We undertook a systematic review of CRXO trials. Searches of MEDLINE, EMBASE, and CINAHL Plus; and citation searches of CRXO methodological articles were conducted to December 2014. We extracted data on design characteristics and statistical methods for sample size, data analysis, and handling of missing data. Results Ninety-one trials including 139 end point analyses met the inclusion criteria. Trials had a median of nine clusters [interquartile range (IQR), 4–21] and median cluster-period size of 30 individuals (IQR, 14–77); 58 (69%) trials had two periods, and 27 trials (30%) included the same individuals in all periods. A rationale for the design was reported in only 25 trials (27%). A sample size justification was provided in 53 (58%) trials. Only nine (10%) trials accounted appropriately for the design in their sample size calculation. Ten of the 12 cluster-level analyses used a method that accounted for the clustering and multiple-period aspects of the design. In contrast, only 4 of the 127 individual-level analyses used a potentially appropriate method. Conclusions There is a need for improved application of appropriate analysis and sample size methods, and reporting, in CRXO trials.

Objectives Adjustment for center in multicenter trials is recommended when there are between-center differences or when randomization has been stratified by center. However, common methods of analysis (such as fixed-effects, Mantel–Haenszel, or stratified Cox models) often require a large number of patients or events per center to perform well. Study Design and Setting We reviewed 206 multicenter randomized trials published in four general medical journals to assess the average number of patients and events per center and determine whether appropriate methods of analysis were used in trials with few patients or events per center. Results The median number of events per center/treatment arm combination for trials using a binary or survival outcome was 3 (interquartile range, 1–10). Sixteen percent of trials had less than 1 event per center/treatment combination, 50% fewer than 3, and 63% fewer than 5. Of the trials which adjusted for center using a method of analysis which requires a large number of events per center, 6% had less than 1 event per center–treatment combination, 25% fewer than 3, and 50% fewer than 5. Methods of analysis that allow for few events per center, such as random-effects models or generalized estimating equations (GEEs), were rarely used. Conclusion Many multicenter trials contain few events per center. Adjustment for center using random-effects models or GEE with model-based (non-robust) standard errors may be beneficial in these scenarios.

Objective To determine whether modifying an outcome definition to remove subjective elements reduced bias in a trial that could not use blinded outcome assessment. Study Design and Setting Reanalysis of an open-label trial comparing a restrictive vs. liberal transfusion strategy for gastrointestinal bleeding. The usual definition of the primary outcome, further bleeding, allows subjective clinical symptoms to be used alone for diagnosis, whereas the definition used in the trial required more objective confirmation by endoscopy. We compared treatment effect estimates for these two definitions. Results Fewer subjective symptom-identified events were confirmed using more objective methods in the restrictive arm (18%) than in the liberal arm (56%), indicating differential assessment between arms. An analysis using all events (both subjective and more objective) led to an odds ratio of 0.83 (95% confidence interval [CI]: 0.50–1.37). When only events confirmed using more objective methods were included, the odds ratio was 0.50 (95% CI: 0.32–0.78). The ratio of the odds ratios was 1.66, indicating that including unconfirmed events in the definition biased the treatment effect upward by 66%. Conclusion Modifying the outcome definition to exclude subjective elements substantially reduced bias. This may be a useful strategy for reducing bias in trials that cannot blind outcome assessment.