Search

It is healthy to periodically question things that we have long taken for granted. Randomized clinical trials are the gold standard for evaluating the benefits and harms of interventions, though often fails to provide the practical evidence to inform medical decision-making. A primary reason is failure to recognize the most important questions for treating patients in clinical practice, and using this as the motivation for the design, monitoring, analysis, and reporting of clinical trials. Our most promising opportunity is placing increased interest on questions of a pragmatic origin to match their clinical importance. Improvements to traditional approaches are needed. Standard approaches synthesizing information obtained from separate marginal analysis of each outcome, fail to incorporate associations between outcomes and recognize the cumulative nature of outcomes in individual patients, suffer from competing risk complexities during interpretation of individual outcomes, and since efficacy and safety analyses are often conducted on different populations, generalizability is unclear. The utility of clinical trials can be enhanced by: (i) using outcomes to analyze patients rather than patients to analyze the outcomes, and (ii) incorporating benefit:risk evaluation into standard trial design and conduct, rather than conducted as a post-hoc exercise. Benefit: risk outcomes such as the desirability of outcome ranking (DOOR), can become a standard, prespecified along with efficacy and safety outcomes. Inclusion and analyses of such endpoints provide important information regarding the effects on patients, unable to be obtained by marginal analyses of efficacy and safety. Prespecified procedures to identify subgroups of patients with a positive benefit:risk profile will further advance clinical trial science. [ABSTRACT FROM AUTHOR]

There is an innate desire to conduct research faster and cheaper. The desire is magnified by todays' business and political pressures. Though understandable, such desires can be dangerous threatening our objectivity and ability to reason, resulting in studies with lower integrity, replicability, and applicability. Modern ideas include: conducting observational rather than randomized studies rationalized by the increasing access to real world data and the belief that modeling can replace randomization, using surrogate rather than clinical outcomes, using surrogate rather than clinical diseases, using PP (Per Protocol) analyses instead of ITT (Intention-to-Treat), using assumptions regarding treatment effects in place of data to estimate those effects, and using adaptive designs that promise efficiencies. These trends are often labeled as "innovation." But is this progress...or regress...a fancy way of lowering the usual evidentiary standard and introducing greater uncertainty? Is it time for a market correction for these "products" that are often presented with a degree of commercialism rather than scientific objectivity? Indeed, our traditional approaches are in need in improvements. Rather than compromising on scientific rigor, can we redirect our motivation to find BETTER answers to the most important questions for patients and clinicians? Though often misunderstood, one such movement is pragmatism, the concept of truly getting closer to understanding the effects of interventions as they are experienced by patients and the value of diagnostics in real world practice. I propose that we place increased interest on questions of a pragmatic origin to match their clinical importance and utility. I present ideas for more pragmatic assessments of the effects of interventions and the yield of diagnostics. [ABSTRACT FROM AUTHOR]

We discuss using prediction as a flexible and practical approach for monitoring futility in clinical trials with two co-primary endpoints (CPE). This approach is appealing in that it provides quantitative evaluation of potential effect sizes and associated precision, and can be combined with flexible error-spending strategies. We extend prediction of effect size estimates and the construction of predicted intervals to the two CPE case, and illustrate interim futility monitoring of treatment effects using prediction with an example. We also discuss alternative approaches based on the conditional and predictive powers, compare these methods and provide some guidance on the use of prediction for better decision in clinical trials with CPE. [ABSTRACT FROM AUTHOR]

I thank and commend Dr. Gibson on his thoughtful and thorough discussion regarding the role of I p i -values in evaluating evidence (Gibson [8]). One could replace hypothesis testing and I p i -values with confidence interval estimation. Observed I p i -values should be reported rather than, for example, I p i < 0.05 or I p i > 0.05. Although a I p i -value of 0.06 and 0.86 each result in a failure to reject a null hypothesis when testing at 0.05, they may be interpreted quite differently in light of other evidence such as results for other outcomes. [Extracted from the article]

We review the design, data monitoring, and analyses of clinical trials with co-primary endpoints. Recently developed methods for fixed-sample and group-sequential settings are described. Practical considerations are discussed, and guidance for the application of these methods is provided. [ABSTRACT FROM PUBLISHER]

Carbapenem-resistant Enterobacteriaceae (CRE) are an emerging and troublesome group of pathogens. Risk factor studies, outcome studies, and randomized trials are three types of studies conducted to answer different types of questions regarding CRE. These studies pose different types of challenges. We discuss issues in the design and analyses of case-control studies, cohort studies, and randomized trials aimed to address various research questions regarding CRE. [ABSTRACT FROM AUTHOR]

In the future, clinical trials will have an increased emphasis on pragmatism, providing a practical description of the effects of new treatments in realistic clinical settings. Accomplishing pragmatism requires better summaries of the totality of the evidence in ways that clinical trials consumers—patients, physicians, insurers—find transparent and allow for informed benefit:risk decision-making. The current approach to the analysis of clinical trials is to analyze efficacy and safety separately and then combine these analyses into a benefit:risk assessment. Many assume that this will effectively describe the impact on patients. But this approach is suboptimal for evaluating the totality of effects on patients. We discuss methods for benefit:risk assessment that have greater pragmatism than methods that separately analyze efficacy and safety. These include the concepts ofwithin-patient analysesand composite benefit:risk endpoints with a goal of understanding how to analyze one patient before trying to figure out how to analyze many. We discuss thedesirability of outcome ranking(DOOR) and introduce thepartial credit strategyusing an example in a clinical trial evaluating the effects of a new antibiotic. As part of the example, we introduce a strategy to engage patients as a resource to inform benefit:risk analyses consistent with the goal of measuring and weighing outcomes that are most important from the patient's perspective. We describe a broad vision for the future of clinical trials consistent with increased pragmatism. Greater focus on using endpoints to analyze patients rather than patients to analyze endpoints particularly in late-phase/stage clinical trials is an important part of this vision. [ABSTRACT FROM PUBLISHER]

The authors comment on the study "Antimicrobial Products: Statistical Challenges and Opportunities" published in the current issue of the journal.

Mixed effects logistic models have become a popular method for analyzing clustered binary data. However, methods to assess goodness of fit are not well developed. Recent work conducted for the ordinary logistic regression model provides the basis for a newly computed mean and variance for the Pearson and the unweighted sums of squares statistics for the mixed effects logistic model. A simulation study was conducted to evaluate the performance of these statistics. The factors that were varied were the number of clusters, the number of observations within a cluster, the magnitude of the correlation, and the number and type of covariates included in the model. Reported simulations results show that the unweighted sums of squares and Pearson statistics have adequate Type I error rates and are recommended for use in evaluating goodness of fit in certain settings. [ABSTRACT FROM AUTHOR]

Generalized Estimating Equations (GEE) have become a popular regression method for analyzing clustered binary data. Methods to assess the goodness of fit of the fitted models have recently been developed. However, published evaluations of these methods under various scenarios are limited. Research conducted for the ordinary logistic regression model provided the basis for a newly computed mean and variance for the Pearson statistic and the unweighted sums of squares statistic for the GEE case. A simulation study was conducted to evaluate the performance of these statistics under various scenarios. The factors that were varied were the number of clusters, the number of observations within a cluster, the magnitude of the correlation, and the number and type of covariates included in the model. Overall, the Pearson and unweighted sums of squares statistics had a satisfactory performance of a type I error rate and are potentially effective in evaluating goodness of fit under certain conditions. [ABSTRACT FROM AUTHOR]

The Toll-like receptor (TLR) adaptor protein MyD88 is integral to airway inflammatory response to microbial- enriched organic dust extract (ODE) exposures. ODE-induced airway neutrophil influx and release of pro-inflammatory cytokines was essentially abrogated in global MyD88-deficient mice, yet these mice demonstrate an increase in airway epithelial cell mucin expression. To further elucidate the role of MyD88-dependent responses specific to lung airway epithelial cells in response to ODE in vivo, the surfactant protein C protein (SPC) Cre+ embryologic expressing airway epithelial cells floxed for MyD88 to disrupt MyD88 signaling were utilized. The inducible club cell secretory protein (CCSP) Cre+, MyD88 floxed, were also developed. Using an established protocol, mice were intranasally instilled with ODE or saline once or daily up to 3weeks. Mice with MyD88-deficient SPC+ lung epithelial cells exhibited decreased neutrophil influx following ODE exposure once and repetitively for 1week without modulation of classic pro-inflammatory mediators including tumor necrosis factor (TNF)-a, interleukin (IL)-6, and neutrophil chemoattractants. This protective response was lost after 3weeks of repetitive exposure. ODE-induced Muc5ac mucin expression at 1 week was also reduced in MyD88-deficient SPC+ cells. Acute ODE-induced IL-33 was reduced in MyD88-deficient SPC+ cells whereas serum IgE levels were increased at one week. In contrast, mice with inducible MyD88-deficient CCSP+ airway epithelial cells demonstrated no significant difference in experimental indices following ODE exposure. Collectively, these findings suggest that MyD88-dependent signaling targeted to all airway epithelial cells plays an important role in mediating neutrophil influx and mucin production in response to acute organic dust exposures. [ABSTRACT FROM AUTHOR]

Clinical patient management is dynamic. It is not based on a single decision but on a sequence of decisions, with adjustments of therapy overtime, where adjustment are personalized to individual patients. However, strategies allowing for such adjustments are infrequently studied. In the treatment of serious bacterial infections, there are two therapeutic decision points: empirical therapy when the patient is first seen by the clinician, and definitive therapy when drug susceptibility results are available to help guide therapeutic choices. COMparing Personalized Antibiotic StrategieS (COMPASS) is a trial design that compares strategies consistent with clinical practice rather than specific treatments. A strategy is a decision-rule guiding treatments of bacterial infections both at the empirical and at the definitive stages. Sequential multiple assignment randomized (SMART) COMPASS allows evaluation of strategies when there are multiple definitive therapy choices. SMART COMPASS is a pragmatic design, mirroring antibiotic treatment decision-making as they unfold in clinical practice and addressing the most relevant question for treating patients: identification of the patient-management strategy that optimizes ultimate patient outcomes. Complex statistical challenges are encountered during the design of SMART COMPASS including how to: appropriately estimate effects and associated standard errors within the context of sequential randomization, identify the best of several strategies while controlling trial-wise error, and calculate sample size and power when trying to identify the optimal strategy. Design considerations for SMART COMPASS are discussed. [ABSTRACT FROM AUTHOR]

We discuss group-sequential three-arm noninferiority clinical trial designs that include active and placebo controls for evaluating both assay sensitivity and noninferiority. We extend two existing approaches, the fixed margin and fraction approaches, into a group-sequential setting with two decision-making frameworks. We investigate the operating characteristics including power, Type I error rate, maximum, and expected sample sizes, as design factors vary. In addition, we discuss sample size recalculation and its impact on the power and Type I error rate via a simulation study. [ABSTRACT FROM PUBLISHER]

Comparing diagnostic tests on accuracy alone can be inconclusive. For example, a test may have better sensitivity than another test yet worse specificity. Comparing tests on benefit risk may be more conclusive because clinical consequences of diagnostic error are considered. For benefit-risk evaluation, we propose diagnostic yield, the expected distribution of subjects with true positive, false positive, true negative, and false negative test results in a hypothetical population. We construct a table of diagnostic yield that includes the number of false positive subjects experiencing adverse consequences from unnecessary work-up. We then develop a decision theory for evaluating tests. The theory provides additional interpretation to quantities in the diagnostic yield table. It also indicates that the expected utility of a test relative to a perfect test is a weighted accuracy measure, the average of sensitivity and specificity weighted for prevalence and relative importance of false positive and false negative testing errors, also interpretable as the cost-benefit ratio of treating non-diseased and diseased subjects. We propose plots of diagnostic yield, weighted accuracy, and relative net benefit of tests as functions of prevalence or cost-benefit ratio. Concepts are illustrated with hypothetical screening tests for colorectal cancer with test positive subjects being referred to colonoscopy. [ABSTRACT FROM PUBLISHER]

When providing advice on a medical product, a major task for FDA advisory committee members is to determine whether or not the benefits of the product under discussion outweigh its risks. Thoughtful and creative utilization of analysis methods and visualization tools often helps the sponsors and the FDA to clearly and succinctly deliver the key message, that is, whether the benefit-risk profile is favorable or unfavorable. In this article, we summarize a review of analysis methods and visual tools that were included in recent (during 2011–2014) FDA advisory committee meeting materials. We summarize the key findings related to benefit-risk (B-R) evaluations. Illustrative examples of analysis and visualization tools are provided along with potential refinements to provide readers' proper context for the appropriate use of various B-R tools in practice when evaluating the B-R balance of medical products. [ABSTRACT FROM PUBLISHER]

The effects of interventions are multidimensional. Use of more than one primary endpoint offers an attractive design feature in clinical trials as they capture more complete characterization of the effects of an intervention and provide more informative intervention comparisons. For these reasons, multiple primary endpoints have become a common design feature in many disease areas such as oncology, infectious disease, and cardiovascular disease. More specifically in medical product development, multiple endpoints are used as co-primary to evaluate the effect of the new interventions. Although methodologies to address continuous co-primary endpoints are well-developed, methodologies for binary endpoints are limited. In this article, we describe power and sample size determination for clinical trials with multiple correlated binary endpoints, when relative risks are evaluated as co-primary. We consider a scenario where the objective is to evaluate evidence for superiority of a test intervention compared with a control intervention, for all of the relative risks. We discuss the normal approximation methods for power and sample size calculations and evaluate how the required sample size, power, and Type I error vary as a function of the correlations among the endpoints. Also we discuss a simple, but conservative procedure for appropriate sample size calculation. We then extend the methods allowing for interim monitoring using group-sequential methods. Supplementary materials for this article are available online. [ABSTRACT FROM PUBLISHER]

We discuss the decision-making frameworks for clinical trials with multiple co-primary endpoints in a group-sequential setting. The decision-making frameworks can account for flexibilities, such as a varying number of analyses, equally or unequally spaced increments of information, and fixed or adaptive Type I error allocation among endpoints. The frameworks can provide efficiency, that is, potentially fewer trial participants, than the fixed sample size designs. We investigate the operating characteristics of the decision-making frameworks and provide guidance on constructing efficient group-sequential strategies in clinical trials with multiple co-primary endpoints. [ABSTRACT FROM AUTHOR]

Sensitivity and specificity are key aspects in evaluating the performance of diagnostic tests. Accuracy and AUC are commonly used composite measures that incorporate sensitivity and specificity. Average Weighted Accuracy (AWA) is motivated by the need for a statistical measure that compares diagnostic tests from the medical costs and clinical impact point of view, while incorporating the relevant prevalence range of the disease as well as the relative importance of false-positive versus false-negative cases. We illustrate the testing procedures in four different scenarios: (i) one diagnostic test vs. the best random test, (ii) two diagnostic tests from two independent samples, (iii) two diagnostic tests from the same sample, and (iv) more than two diagnostic tests from different or the same samples. The impacts of sample size, prevalence, and relative importance on power and average medical costs/clinical loss are examined through simulation studies. Accuracy has the highest power while AWA provides a consistent criterion in selecting the optimal threshold and better diagnostic tests with direct clinical interpretations. The use of AWA is illustrated on a three-arm clinical trial evaluating three different assays in detecting Neisseria gonorrhoeae and Chlamydia trachomatis in the rectum and pharynx. [ABSTRACT FROM AUTHOR]