Search

In adaptive clinical trials, the conventional confidence interval (CI) for a treatment effect is prone to undesirable properties such as undercoverage and potential inconsistency with the final hypothesis testing decision. Accordingly, as is stated in recent regulatory guidance on adaptive designs, there is the need for caution in the interpretation of CIs constructed during and after an adaptive clinical trial. However, it may be unclear which of the available CIs in the literature are preferable. This paper is the second in a two-part series that explores CIs for adaptive trials. Part I provided a methodological review of approaches to construct CIs for adaptive designs. In this paper (part II), we present an extended case study based around a two-stage group sequential trial, including a comprehensive simulation study of the proposed CIs for this setting. This facilitates an expanded description of considerations around what makes for an effective CI procedure following an adaptive trial. We show that the CIs can have notably different properties. Finally, we propose a set of guidelines for researchers around the choice of CIs and the reporting of CIs following an adaptive design., Comment: 40 pages

Regulatory guidance notes the need for caution in the interpretation of confidence intervals (CIs) constructed during and after an adaptive clinical trial. Conventional CIs of the treatment effects are prone to undercoverage (as well as other undesirable properties) in many adaptive designs, because they do not take into account the potential and realised trial adaptations. This paper is the first in a two-part series that explores CIs for adaptive trials. It provides a comprehensive review of the methods to construct CIs for adaptive designs, while the second paper illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. We describe several classes of techniques for constructing CIs for adaptive clinical trials, before providing a systematic literature review of available methods, classified by the type of adaptive design. As part of this, we assess, through a proposed traffic light system, which of several desirable features of CIs (such as achieving nominal coverage and consistency with the hypothesis test decision) each of these methods holds., Comment: 29 pages

Under a generalised estimating equation analysis approach, approximate design theory is used to determine Bayesian D-optimal designs. For two examples, considering simple exchangeable and exponential decay correlation structures, we compare the efficiency of identified optimal designs to balanced stepped-wedge designs and corresponding stepped-wedge designs determined by optimising using a normal approximation approach. The dependence of the Bayesian D-optimal designs on the assumed correlation structure is explored; for the considered settings, smaller decay in the correlation between outcomes across time periods, along with larger values of the intra-cluster correlation, leads to designs closer to a balanced design being optimal. Unlike for normal data, it is shown that the optimal design need not be centro-symmetric in the binary outcome case. The efficiency of the Bayesian D-optimal design relative to a balanced design can be large, but situations are demonstrated in which the advantages are small. Similarly, the optimal design from a normal approximation approach is often not much less efficient than the Bayesian D-optimal design. Bayesian D-optimal designs can be readily identified for stepped-wedge cluster randomised trials with binary outcome data. In certain circumstances, principally ones with strong time period effects, they will indicate that a design unlikely to have been identified by previous methods may be substantially more efficient. However, they require a larger number of assumptions than existing optimal designs, and in many situations existing theory under a normal approximation will provide an easier means of identifying an efficient design for binary outcome data.

Adaptive designs(AD) are a broad class of trial designs that allow preplanned modifications based on patient data providing improved efficiency and flexibility. However, a delay in observing the primary outcome variable can harm this added efficiency. In this paper, we aim to ascertain the size of such outcome delay that results in the realised efficiency gains of ADs becoming negligible compared to classical fixed sample RCTs. We measure the impact of delay by developing formulae for the no. of overruns in 2 arm GSDs with normal data, assuming different recruitment models. The efficiency of a GSD is usually measured in terms of the expected sample size (ESS), with GSDs generally reducing the ESS compared to a standard RCT. Our formulae measures the efficiency gain from a GSD in terms of ESS reduction that is lost due to delay. We assess whether careful choice of design (e.g., altering the spacing of the IAs) can help recover the benefits of GSDs in presence of delay. We also analyse the efficiency of GSDs with respect to time to complete the trial. Comparing the expected efficiency gains, with and without consideration of delay, it is evident GSDs suffer considerable losses due to delay. Even a small delay can have a significant impact on the trial's efficiency. In contrast, even in the presence of substantial delay, a GSD will have a smaller expected time to trial completion in comparison to a simple RCT. Although the no. of stages have little influence on the efficiency losses, the timing of IAs can impact the efficiency of a GSDs with delay. Particularly, for unequally spaced IAs, pushing IAs towards latter end of the trial can be harmful for the design with delay.

Introduction: Even in effectively conducted randomised trials, the probability of a successful study remains relatively low. With recent advances in the next-generation sequencing technologies, there is a rapidly growing number of high-dimensional data, including genetic, molecular and phenotypic information, that have improved our understanding of driver genes, drug targets, and drug mechanisms of action. The leveraging of high-dimensional data holds promise for increased success of clinical trials. Methods: We provide an overview of methods for utilising high-dimensional data in clinical trials. We also investigate the use of these methods in practice through a review of recently published randomised clinical trials that utilise high-dimensional genetic data. The review includes articles that were published between 2019 and 2021, identified through the PubMed database. Results: Out of 174 screened articles, 100 (57.5%) were randomised clinical trials that collected high-dimensional data. The most common clinical area was oncology (30%), followed by chronic diseases (28%), nutrition and ageing (18%) and cardiovascular diseases (7%). The most common types of data analysed were gene expression data (70%), followed by DNA data (21%). The most common method of analysis (36.3%) was univariable analysis. Articles that described multivariable analyses used standard statistical methods. Most of the clinical trials had two arms. Discussion: New methodological approaches are required for more efficient analysis of the increasing amount of high-dimensional data collected in randomised clinical trials. We highlight the limitations and barriers to the current use of high-dimensional data in trials, and suggest potential avenues for improvement and future work., Comment: 17 pages, 3 figures, 2 tables

Basket trials are increasingly used for the simultaneous evaluation of a new treatment in various patient subgroups under one overarching protocol. We propose a Bayesian approach to sample size determination in basket trials that permit borrowing of information between commensurate subsets. Specifically, we consider a randomised basket trial design where patients are randomly assigned to the new treatment or a control within each trial subset (`subtrial' for short). Closed-form sample size formulae are derived to ensure each subtrial has a specified chance of correctly deciding whether the new treatment is superior to or not better than the control by some clinically relevant difference. Given pre-specified levels of pairwise (in)commensurability, the subtrial sample sizes are solved simultaneously. The proposed Bayesian approach resembles the frequentist formulation of the problem in yielding comparable sample sizes for circumstances of no borrowing. When borrowing is enabled between commensurate subtrials, a considerably smaller trial sample size is required compared to the widely implemented approach of no borrowing. We illustrate the use of our sample size formulae with two examples based on real basket trials. A comprehensive simulation study further shows that the proposed methodology can maintain the true positive and false positive rates at desired levels., Comment: A submitted paper with 18 pages (inclusive of 3 figures and 1 table) and 9 pages of Supplementary Materials

Existing multi-outcome designs focus almost entirely on evaluating whether all outcomes show evidence of efficacy or whether at least one outcome shows evidence of efficacy. While a small number of authors have provided multi-outcome designs that evaluate when a general number of outcomes show promise, these designs have been single-stage in nature only. We therefore propose two designs, of group-sequential and drop the loser form, that provide this design characteristic in a multi-stage setting. Previous such multi-outcome multi-stage designs have allowed only for a maximum of two outcomes; our designs thus also extend previous related proposals by permitting any number of outcomes., Comment: 38 pages, 11 figures, 4 tables. Submitted for the degree of Doctor of Philosophy at University of Cambridge

Adapting the final sample size of a trial to the evidence accruing during the trial is a natural way to address planning uncertainty. Designs with adaptive sample size need to account for their optional stopping to guarantee strict type-I error-rate control. A variety of different methods to maintain type-I error-rate control after unplanned changes of the initial sample size have been proposed in the literature. This makes interim analyses for the purpose of sample size recalculation feasible in a regulatory context. Since the sample size is usually determined via an argument based on the power of the trial, an interim analysis raises the question of how the final sample size should be determined conditional on the accrued information. Conditional power is a concept often put forward in this context. Since it depends on the unknown effect size, we take a strict estimation perspective and compare assumed conditional power, observed conditional power, and predictive power with respect to their properties as estimators of the unknown conditional power. We then demonstrate that pre-planning an interim analysis using methodology for unplanned interim analyses is ineffective and naturally leads to the concept of optimal two-stage designs. We conclude that unplanned design adaptations should only be conducted as reaction to trial-external new evidence, operational needs to violate the originally chosen design, or post hoc changes in the objective criterion. Finally, we show that commonly discussed sample size recalculation rules can lead to paradoxical outcomes and propose two alternative ways of reacting to newly emerging trial-external evidence.

Purpose: Two-stage single-arm trial designs are commonly used in phase II oncology to infer treatment effects for a binary primary outcome (e.g., tumour response). It is imperative that such studies be designed, analysed, and reported effectively. However, there is little available evidence on whether this is the case, particularly for key statistical considerations. We therefore comprehensively review such trials, examining in particular quality of reporting. Methods: Published oncology trials that utilised "Simon's two-stage design" over a 5 year period were identified and reviewed. Articles were evaluated on whether they reported sufficient design details, such as the required sample size, and analysis details, such as a confidence interval (CI). The articles that did not adjust their inference for the incorporation of an interim analysis were re-analysed to evaluate the impact on their reported point estimate and CI. Results: Four hundred and twenty five articles that reported the results of a single treatment arm were included. Of these, 47.5% provided the five components that ensure design reproducibility. Only 1.2% and 2.1% reported an adjusted point estimate or CI, respectively. Just 55.3% of trials provided the final stage rejection bound, indicating many trials did not test the hypothesis the design is constructed to assess. Re-analysis of the trials suggests that reported point estimates underestimated treatment effects and that reported CIs were too narrow. Conclusion: Key design details of two-stage single-arm trials are often unreported. Whilst inference is regularly performed, it is rarely done so in a way that removes the bias introduced by the interim analysis. In order to maximise their value, future studies must improve the way that they are analysed and reported.

Sample size derivation is a crucial element of the planning phase of any confirmatory trial. A sample size is typically derived based on constraints on the maximal acceptable type I error rate and a minimal desired power. Here, power depends on the unknown true effect size. In practice, power is typically calculated either for the smallest relevant effect size or a likely point alternative. The former might be problematic if the minimal relevant effect is close to the null, thus requiring an excessively large sample size. The latter is dubious since it does not account for the a priori uncertainty about the likely alternative effect size. A Bayesian perspective on the sample size derivation for a frequentist trial naturally emerges as a way of reconciling arguments about the relative a priori plausibility of alternative effect sizes with ideas based on the relevance of effect sizes. Many suggestions as to how such `hybrid' approaches could be implemented in practice have been put forward in the literature. However, key quantities such as assurance, probability of success, or expected power are often defined in subtly different ways in the literature. Starting from the traditional and entirely frequentist approach to sample size derivation, we derive consistent definitions for the most commonly used `hybrid' quantities and highlight connections, before discussing and demonstrating their use in the context of sample size derivation for clinical trials.

In single-arm phase II oncology trials, the most popular choice of design is Simon's two-stage design, which allows early stopping at one interim analysis. However, the expected trial sample size can be reduced further by allowing curtailment. Curtailment is stopping when the final go or no-go decision is certain, so-called non-stochastic curtailment, or very likely, known as stochastic curtailment. In the context of single-arm phase II oncology trials, stochastic curtailment has previously been restricted to stopping in the second stage and/or stopping for a no-go decision only. We introduce two designs that incorporate stochastic curtailment and allow stopping after every observation, for either a go or no-go decision. We obtain optimal stopping boundaries by searching over a range of potential conditional powers, beyond which the trial will stop for a go or no-go decision. This search is novel: firstly, the search is undertaken over a range of values unique to each possible design realisation. Secondly, these values are evaluated taking into account the possibility of early stopping. Finally, each design realisation's operating characteristics are obtained exactly. The proposed designs are compared to existing designs in a real data example. They are also compared under three scenarios, both with respect to four single optimality criteria and using a loss function. The proposed designs are superior in almost all cases. Optimising for the expected sample size under either the null or alternative hypothesis, the saving compared to the popular Simon's design ranges from 22% to 55%.

Multi-arm designs provide an effective means of evaluating several treatments within the same clinical trial. Given the large number of treatments now available for testing in many disease areas, it has been argued that their utilisation should increase. However, for any given clinical trial there are numerous possible multi-arm designs that could be used, and choosing between them can be a difficult task. This task is complicated further by a lack of available easy-to-use software for designing multi-arm trials. To aid the wider implementation of multi-arm clinical trial designs, we have developed a web application for sample size calculation when using a variety of popular multiple comparison corrections. Furthermore, the application supports sample size calculation to control several varieties of power, as well as the determination of optimised arm-wise allocation ratios. It is built using the Shiny package in the R programming language, is free to access on any device with an internet browser, and requires no programming knowledge to use. The application provides the core information required by statisticians and clinicians to review the operating characteristics of a chosen multi-arm clinical trial design. We hope that it will assist with the future utilisation of such designs in practice.

Background/Aims: The increasing expense of the drug development process has seen interest in the use of adaptive designs (ADs) grow substantially in recent years. Accordingly, much research has been conducted to identify potential barriers to increasing the use of ADs in practice, and several articles have argued that the availability of user-friendly software will be an important step in making ADs easier to implement. Therefore, in this paper we present a review of the current state of software availability for AD. Methods: We first review articles from 31 journals published in 2013-17 that relate to methodology for adaptive trials, in order to assess how often code and software for implementing novel ADs is made available at the time of publication. We contrast our findings against these journals' current policies on code distribution. Secondly, we conduct additional searches of popular code repositories, such as CRAN and GitHub, to identify further existing user-contributed software for ADs. From this, we are able to direct interested parties towards solutions for their problem of interest by classifying available code by type of adaptation. Results: Only 29% of included articles made their code available in some form. In many instances, articles published in journals that had mandatory requirements on code provision still did not make code available. There are several areas in which available software is currently limited or saturated. In particular, many packages are available to address group sequential design, but comparatively little code is present in the public domain to determine biomarker-guided ADs. Conclusions: There is much room for improvement in the provision of software alongside AD publications. Additionally, whilst progress has been made, well-established software for various types of trial adaptation remains sparsely available.

The determination of the sample size required by a crossover trial typically depends on the specification of one or more variance components. Uncertainty about the value of these parameters at the design stage means that there is often a risk a trial may be under- or over-powered. For many study designs, this problem has been addressed by considering adaptive design methodology that allows for the re-estimation of the required sample size during a trial. Here, we propose and compare several approaches for this in multi-treatment crossover trials. Specifically, regulators favour re-estimation procedures to maintain the blinding of the treatment allocations. We therefore develop blinded estimators for the within and between person variances, following simple or block randomisation. We demonstrate that, provided an equal number of patients are allocated to sequences that are balanced for period, the proposed estimators following block randomisation are unbiased. We further provide a formula for the bias of the estimators following simple randomisation. The performance of these procedures, along with that of an unblinded approach, is then examined utilising three motivating examples, including one based on a recently completed four-treatment four-period crossover trial. Simulation results show that the performance of the proposed blinded procedures is in many cases similar to that of the unblinded approach, and thus they are an attractive alternative.

Recently, methodology was presented to facilitate the incorporation of interim analyses in stepped-wedge (SW) cluster randomised trials (CRTs). Here, we extend this previous discussion. We detail how the stopping boundaries, allocation sequences, and per-cluster per-period sample size of a group sequential SW-CRT can be optimised. We then describe methods by which point estimates, p-values, and confidence intervals, which account for the sequential nature of the design, can be calculated. We demonstrate that optimal sequential designs can reduce the expected required number of measurements under the null hypothesis, compared to the classical design, by up to 30%, with no cost to the maximal possible required number of measurements. Furthermore, the adjusted analysis procedure almost universally reduces the average bias in the point estimate, and consistently provides a confidence interval with coverage close to the nominal level. In contrast, the coverage of a naive 95% confidence interval is observed to range between 92 and 98%. Methodology is now readily available for the efficient design and analysis of group sequential SW-CRTs. In scenarios in which there are substantial ethical or financial reasons to terminate a SW-CRT as soon as possible, trialists should strongly consider a group sequential approach.

Primarily motivated by the drug development process, several publications have now presented methodology for the design of multi-arm multi-stage experiments with normally distributed outcome variables of known variance. Here, we extend these past considerations to allow the design of what we refer to as an abcd multi-arm multi-stage experiment. We provide a proof of how strong control of the a-generalised type-I familywise error-rate can be ensured. We then describe how to attain the power to reject at least b out of c false hypotheses, which is related to controlling the b-generalised type-II familywise error-rate. Following this, we detail how a design can be optimised for a scenario in which rejection of any d null hypotheses brings about termination of the experiment. We achieve this by proposing a highly computationally efficient approach for evaluating the performance of a candidate design. Finally, using a real clinical trial as a motivating example, we explore the effect of the design's control parameters on the statistical operating characteristics.

In small sample studies with binary outcome data, use of a normal approximation for hypothesis testing can lead to substantial inflation of the type-I error-rate. Consequently, exact statistical methods are necessitated, and accordingly, much research has been conducted to facilitate this. Recently, this has included methodology for the design of two-stage multi-arm studies utilising exact binomial tests. These designs were demonstrated to carry substantial efficiency advantages over a fixed sample design, but generally suffered from strong conservatism. An alternative classical means of small sample inference with dichotomous data is Fisher's exact test. However, this method is limited to single-stage designs when there are multiple arms. Therefore, here, we propose a two-stage version of Fisher's exact test, with the potential to stop early to accept or reject null hypotheses, which is applicable to multi-arm studies. In particular, we provide precise formulae describing the requirements for achieving weak or strong control of the familywise error-rate with this design. Following this, we describe how the design parameters may be optimised to confer desirable operating characteristics. For a motivating example based on a phase II clinical trial, we demonstrate that on average our approach is less conservative than corresponding optimal designs based on exact binomial tests.

This paper presents a set of Stata commands and Mata functions to evaluate different distributional quantities of the multivariate normal distribution, and a particular type of non-central multivariate t distribution. Specifically, their densities, distribution functions, equicoordinate quantiles, and pseudo-random vectors can be computed efficiently, either in the absence or presence of variable truncation.

Numerous publications have now addressed the principles of designing, analyzing, and reporting the results of, stepped-wedge cluster randomized trials. In contrast, there is little research available pertaining to the design and analysis of multi-arm stepped-wedge cluster randomized trials, utilized to evaluate the effectiveness of multiple experimental interventions. In this paper, we address this by explaining how the required sample size in these multi-arm trials can be ascertained when data are to be analyzed using a linear mixed model. We then go on to describe how the design of such trials can be optimized to balance between minimizing the cost of the trial, and minimizing some function of the covariance matrix of the treatment effect estimates. Using a recently commenced trial that will evaluate the effectiveness of sensor monitoring in an occupational therapy rehabilitation program for older persons after hip fracture as an example, we demonstrate that our designs could reduce the number of observations required for a fixed power level by up to 58%. Consequently, when logistical constraints permit the utilization of any one of a range of possible multi-arm stepped-wedge cluster randomized trial designs, researchers should consider employing our approach to optimize their trials efficiency.

Crossover designs are an extremely useful tool to investigators, whilst group sequential methods have proven highly proficient at improving the efficiency of parallel group trials. Yet, group sequential methods and crossover designs have rarely been paired together. One possible explanation for this could be the absence of a formal proof of how to strongly control the familywise error rate in the case when multiple comparisons will be made. Here, we provide this proof, valid for any number of initial experimental treatments and any number of stages, when results are analysed using a linear mixed model. We then establish formulae for the expected sample size and expected number of observations of such a trial, given any choice of stopping boundaries. Finally, utilising the four-treatment, four-period TOMADO trial as an example, we demonstrate group sequential methods in this setting could have reduced the trials expected number of observations under the global null hypothesis by over 33%.

Multi-arm multi-stage trial designs can bring notable gains in efficiency to the drug development process. However, for normally distributed endpoints, the determination of a design typically depends on the assumption that the patient variance in response is known. In practice, this will not usually be the case. To allow for unknown variance, previous research explored the performance of t-test statistics, coupled with a quantile substitution procedure for modifying the stopping boundaries, at controlling the familywise error-rate to the nominal level. Here, we discuss an alternative method based on Monte Carlo simulation that allows the group size and stopping boundaries of a multi-arm multi-stage t-test to be optimised according to some nominated optimality criteria. We consider several examples, provide R code for general implementation, and show that our designs confer a familywise error-rate and power close to the desired level. Consequently, this methodology will provide utility in future multi-arm multi-stage trials.

In a group sequential clinical trial, accumulated data are analysed at numerous time-points in order to allow early decisions about a hypothesis of interest. These designs have historically been recommended for their ethical, administrative and economic benefits. In this work, we discuss a collection of new Stata commands for computing the stopping boundaries and required group size of various classical group sequential designs, assuming a normally distributed outcome variable. Following this, we demonstrate how the performance of several designs can be compared graphically.

The ability to accurately estimate the sample size required by a stepped-wedge (SW) cluster randomized trial (CRT) routinely depends upon the specification of several nuisance parameters. If these parameters are mis-specified, the trial could be over-powered, leading to increased cost, or under-powered, enhancing the likelihood of a false negative. We address this issue here for cross-sectional SW-CRTs, analyzed with a particular linear mixed model, by proposing methods for blinded and unblinded sample size re-estimation (SSRE). Blinded estimators for the variance parameters of a SW-CRT analyzed using the Hussey and Hughes model are derived. Then, procedures for blinded and unblinded SSRE after any time period in a SW-CRT are detailed. The performance of these procedures is then examined and contrasted using two example trial design scenarios. We find that if the two key variance parameters were under-specified by 50%, the SSRE procedures were able to increase power over the conventional SW-CRT design by up to 29%, resulting in an empirical power above the desired level. Moreover, the performance of the re-estimation procedures was relatively insensitive to the timing of the interim assessment. Thus, the considered SSRE procedures can bring substantial gains in power when the underlying variance parameters are mis-specified. Though there are practical issues to consider, the procedure's performance means researchers should consider incorporating SSRE in to future SW-CRTs.