Search

Background and objective Novel cancer therapies are often approved with evidence from a single pivotal trial alone. There are concerns about the credibility of this evidence. Higher validity may be indicated by five methodological and statistical characteristics of pivotal trial evidence that were described by the U.S. Food and Drug Administration (FDA), which may corroborate the reliance on a single trial alone for approval decisions. Study design We did a metaepidemiologic evaluation of all single pivotal trials supporting FDA approval of novel drugs and therapeutic biologicals for cancers between 2000 and 2016. For each trial, we determined the presence of these five characteristics, which we operationalized as (1) large and multicenter trial (≥200 patients; more than one center); consistent treatment benefits across (2) multiple patient subgroups (in view of FDA reviewers), (3) multiple endpoints (including overall survival, progression-free survival, response rate, health related quality of life), and (4) multiple treatment comparisons (e.g., multi-arm studies); and (5) “statistically very persuasive” results (P-values Results Thirty-five of 100 approvals were based on evidence from a single pivotal trial without any further supporting evidence on beneficial effects (20 randomized controlled trials and 15 single-arm trials). The number increased substantially from one approval before 2006 to 23 after 2011. Sixty-six percent (23/35) of the trials were large multicenter trials (median 301 patients and 63 centers). Consistent effects were demonstrated across subgroups in 66% (23/35), across endpoints in 43% (15/35), and across multiple comparisons in 3% (1/35). Very lowP-values for the primary endpoint were seen in 34% (12/35). At least one of the corroborating characteristics was present in 94% (33/35) of all approvals, two or more were present in 54% (19/35), and none had all characteristics. Conclusions Single pivotal trials typically have some of the corroborating characteristics, but often only one or two. These characteristics need to be better operationalized, defined, and reported and whether single trials with such characteristics provide similar evidence about benefits and harms of novel treatments as multiple trials would do needs to be shown.

Objectives Off-label drug use is highly prevalent but controversial and often discouraged assuming generally inferior medical effects associated with off-label use. Study Design and Setting We searched PubMed, MEDLINE, PubMed Health, and the Cochrane Library up to May 2015 for systematic reviews including meta-analyses of randomized clinical trials (RCTs) comparing off-label and approved drugs head-to-head in any population and on any medical outcome. We combined the comparative effects in meta-analyses providing summary odds ratios (sOR) for each treatment comparison and outcome, and then calculated an overall summary of the sOR across all comparisons (ssOR). Results We included 25 treatment comparisons with 153 RCTs and 24,592 patients. In six of 25 comparisons (24%), off-label drugs were significantly superior (five of 25) or inferior (one of 25) to approved treatments. There was substantial statistical heterogeneity across comparisons (I2 = 43%). Overall, off-label drugs were more favorable than approved treatments (ssOR 0.72; 95% CI = 0.54–0.95). Analyses of patient-relevant outcomes were similar (statistical significant differences in 24% (six of 25); ssOR 0.74; 95% CI = 0.56–0.98; I2 = 60%). Analyses of primary outcomes of the systematic reviews (n = 22 comparisons) indicated less heterogeneity and no statistically significant difference overall (ssOR 0.85; 95% CI = 0.67–1.06; I2 = 0%). Conclusion Approval status does not reliably indicate which drugs are more favorable in situations with clinical trial evidence comparing off-label with approved use. Drug effectiveness assessments without considering off-label use may provide incomplete information. To ensure that patients receive the best available care, funding, policy, reimbursement, and treatment decisions should be evidence based considering the entire spectrum of available therapeutic choices.

Objectives To evaluate how estimated treatment effects agree between nonrandomized studies using causal modeling with marginal structural models (MSM-studies) and randomized trials (RCTs). Study Design Meta-epidemiological study. Setting MSM-studies providing effect estimates on any healthcare outcome of any treatment were eligible. We systematically sought RCTs on the same clinical question and compared the direction of treatment effects, effect sizes, and confidence intervals. Results The main analysis included 19 MSM-studies (1,039,570 patients) and 141 RCTs (120,669 patients). MSM-studies indicated effect estimates in the opposite direction from RCTs for eight clinical questions (42%), and their 95% CI (confidence interval) did not include the RCT estimate in nine clinical questions (47%). The effect estimates deviated 1.58-fold between the study designs (median absolute deviation OR [odds ratio] 1.58; IQR [interquartile range] 1.37 to 2.16). Overall, we found no systematic disagreement regarding benefit or harm but confidence intervals were wide (summary ratio of odds ratios [sROR] 1.04; 95% CI 0.88 to 1.23). The subset of MSM-studies focusing on healthcare decision-making tended to overestimate experimental treatment benefits (sROR 1.44; 95% CI 0.99 to 2.09). Conclusion Nonrandomized studies using causal modeling with MSM may give different answers than RCTs. Caution is still required when nonrandomized “real world” evidence is used for healthcare decisions.

Objectives To determine how marginal structural models (MSMs), which are increasingly used to estimate causal effects, are used in randomized clinical trials (RCTs) and compare their results with those from intention-to-treat (ITT) or other analyses. Study Design and Setting We searched PubMed, Scopus, citations of key references, and Clinicaltrials.gov. Eligible RCTs reported clinical effects based on MSMs and at least one other analysis. Results We included 12 RCTs reporting 138 analyses for 24 clinical questions. In 19/24 (79%), MSM-based and other effect estimates were all in the same direction, 22/22 had overlapping 95% confidence intervals (CIs), and in 19/22 (86%), the MSM effect estimate lay within all 95% CIs of all other effects (in two cases no CIs were reported). For the same clinical question, the largest effect estimate from any analysis was 1.19-fold (median; interquartile range 1.13-1.34) larger than the smallest. All MSM and ITT effect estimates were in the same direction and had overlapping 95% CIs. In 71% (12/17), they also agreed on the presence of statistical significance. MSM-based effect estimates deviated more from the null than those based on ITT (P = 0.18). The effect estimates of both approaches differed 1.12-fold (median; interquartile range 1.02-1.22). Conclusions MSMs provided largely similar effect estimates as other available analyses. Nevertheless, some of the differences in effect estimates or statistical significance may become important in clinical decision-making, and the multiple estimates require utmost attention of possible selective reporting bias.

When little or no data directly comparing two treatments are available, investigators often rely on indirect comparisons from studies testing the treatments against a control or placebo. One approach to indirect comparison is to pool findings from the active treatment arms of the original controlled trials. This approach offers no advantage over a comparison of observational study data and is prone to bias. We present an alternative model that evaluates the differences between treatment and placebo in two sets of clinical trials, and preserves the randomization of the originally assigned patient groups. We apply the method to data on sulphamethoxazole-trimethoprim or dapsone/pyrimethamine as prophylaxis against Pneumocystis cari- nii in HIV infected patients. The indirect comparison showed substantial increased benefit from the former (odds ratio 0.37, 95% Cl 0.21 to 0.65), while direct comparisons from randomized trials suggests a much smaller difference (risk ratio 0.64, 95% CI 0.45 to 0.90; p-value for difference of effect = 0.11). Direct comparisons of treatments should be sought. When direct comparisons are unavailable, indirect compar- ison meta-analysis should evaluate the magnitude of treatment effects across studies, recognizing the limited strength of inference. J CLIN EPIDEMIOL 50;6:683-691, 1997. 0 1997 Elsevier Science Inc.

OBJECTIVE: Hierarchical modeling has been proposed as a solution to the multiple exposure problem. We estimate associations between metabolic syndrome and different components of antiretroviral therapy using both conventional and hierarchical models. STUDY DESIGN AND SETTING: We use discrete time survival analysis to estimate the association between metabolic syndrome and cumulative exposure to 16 antiretrovirals from four drug classes. We fit a hierarchical model where the drug class provides a prior model of the association between metabolic syndrome and exposure to each antiretroviral. RESULTS: One thousand two hundred and eighteen patients were followed for a median of 27 months, with 242 cases of metabolic syndrome (20%) at a rate of 7.5 cases per 100 patient years. Metabolic syndrome was more likely to develop in patients exposed to stavudine, but was less likely to develop in those exposed to atazanavir. The estimate for exposure to atazanavir increased from hazard ratio of 0.06 per 6 months' use in the conventional model to 0.37 in the hierarchical model (or from 0.57 to 0.81 when using spline-based covariate adjustment). CONCLUSION: These results are consistent with trials that show the disadvantage of stavudine and advantage of atazanavir relative to other drugs in their respective classes. The hierarchical model gave more plausible results than the equivalent conventional model.

In a randomized double-blind trial 251 adults with sinusitis-like symptoms were given amoxicillin/clavulanate or placebo for 6 days. Seven diagnostic indicators for acute bacterial rhinosinusitis are compared by their accuracy assuming a latent class model and by the treatment effect that they would have had if used to select a subset of patients for antibiotic treatment. Under a latent class model, radiography is a more efficient indicator then C reactive protein (CRP), which is, in turn, more efficient than other clinical signs and symptoms. However, a history of purulent nasal discharge, and signs of pus in the nasal cavity and throat, are better criteria than radiography or CRP for selecting those patients who will benefit from antibiotic treatment. These contradictory results are a salutary reminder that diagnostic indicators need to be evaluated in terms of therapeutic consequences for the patient.

Objectives To investigate the frequency of interim analyses, stopping rules, and data safety and monitoring boards (DSMBs) in protocols of randomized controlled trials (RCTs); to examine these features across different reasons for trial discontinuation; and to identify discrepancies in reporting between protocols and publications. Study Design and Setting We used data from a cohort of RCT protocols approved between 2000 and 2003 by six research ethics committees in Switzerland, Germany, and Canada. Results Of 894 RCT protocols, 289 prespecified interim analyses (32.3%), 153 stopping rules (17.1%), and 257 DSMBs (28.7%). Overall, 249 of 894 RCTs (27.9%) were prematurely discontinued; mostly due to reasons such as poor recruitment, administrative reasons, or unexpected harm. Forty-six of 249 RCTs (18.4%) were discontinued due to early benefit or futility; of those, 37 (80.4%) were stopped outside a formal interim analysis or stopping rule. Of 515 published RCTs, there were discrepancies between protocols and publications for interim analyses (21.1%), stopping rules (14.4%), and DSMBs (19.6%). Conclusion Two-thirds of RCT protocols did not consider interim analyses, stopping rules, or DSMBs. Most RCTs discontinued for early benefit or futility were stopped without a prespecified mechanism. When assessing trial manuscripts, journals should require access to the protocol.