Search

Competing Interests: Competing interests: All authors have completed the ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare: support from the UK Medical Research Council and the NIHR for the submitted work. BCK and SC declare grant funding (payable to employing institutions) from the MRC-NIHR Trials Methodology Research Partnership. BCK and ME declare grant funding (payable to employing institutions) from the NIHR for the FLO-ELA trial. TPM declares consultancy fees from Bayer Healthcare Pharmaceuticals, Alliance Pharmaceuticals, Gilead Sciences, and Kite Pharma; declares conference attendance and travel paid for as an invited speaker at the 2023 European Society for Blood and Marrow Transplantation conference; and is an independent member of the data and safety monitoring board for the FLO-ELA trial. All other authors declare no conflicts of interest.

Importance: Transparent reporting of randomized trials is essential to facilitate critical appraisal and interpretation of results. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, reporting of factorial trials is suboptimal., Objective: To develop a consensus-based extension to the Consolidated Standards of Reporting Trials (CONSORT) 2010 Statement for factorial trials., Design: Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the CONSORT extension for factorial trials was developed by (1) generating a list of reporting recommendations for factorial trials using a scoping review of methodological articles identified using a MEDLINE search (from inception to May 2019) and supplemented with relevant articles from the personal collections of the authors; (2) a 3-round Delphi survey between January and June 2022 to identify additional items and assess the importance of each item, completed by 104 panelists from 14 countries; and (3) a hybrid consensus meeting attended by 15 panelists to finalize the selection and wording of items for the checklist., Findings: This CONSORT extension for factorial trials modifies 16 of the 37 items in the CONSORT 2010 checklist and adds 1 new item. The rationale for the importance of each item is provided. Key recommendations are (1) the reason for using a factorial design should be reported, including whether an interaction is hypothesized, (2) the treatment groups that form the main comparisons should be clearly identified, and (3) for each main comparison, the estimated interaction effect and its precision should be reported., Conclusions and Relevance: This extension of the CONSORT 2010 Statement provides guidance on the reporting of factorial randomized trials and should facilitate greater understanding of and transparency in their reporting.

Importance: Trial protocols outline a trial's objectives as well as the methods (design, conduct, and analysis) that will be used to meet those objectives, and transparent reporting of trial protocols ensures objectives are clear and facilitates appraisal regarding the suitability of study methods. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, no extension of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 Statement, which provides guidance on reporting of trial protocols, for factorial trials is available., Objective: To develop a consensus-based extension to the SPIRIT 2013 Statement for factorial trials., Evidence Review: The SPIRIT extension for factorial trials was developed using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework. First, a list of reporting recommendations was generated using a scoping review of methodological articles identified using a MEDLINE search (inception to May 2019), which was supplemented with relevant articles from the personal collections of the authors. Second, a 3-round Delphi survey (January to June 2022, completed by 104 panelists from 14 countries) was conducted to assess the importance of items and identify additional recommendations. Third, a hybrid consensus meeting was held, attended by 15 panelists to finalize selection and wording of the checklist., Findings: This SPIRIT extension for factorial trials modified 9 of the 33 items in the SPIRIT 2013 checklist. Key reporting recommendations were that the rationale for using a factorial design should be provided, including whether an interaction is hypothesized; the treatment groups that will form the main comparisons should be identified; and statistical methods for each main comparison should be provided, including how interactions will be assessed., Conclusions and Relevance: In this consensus statement, 9 factorial-specific items were provided that should be addressed in all protocols of factorial trials to increase the trial's utility and transparency.

Background: Recent work has shown that cluster-randomised trials can estimate two distinct estimands: the participant-average and cluster-average treatment effects. These can differ when participant outcomes or the treatment effect depends on the cluster size (termed informative cluster size). In this case, estimators that target one estimand (such as the analysis of unweighted cluster-level summaries, which targets the cluster-average effect) may be biased for the other. Furthermore, commonly used estimators such as mixed-effects models or generalised estimating equations with an exchangeable correlation structure can be biased for both estimands. However, there has been little empirical research into whether informative cluster size is likely to occur in practice., Method: We re-analysed a cluster-randomised trial comparing two different thresholds for red blood cell transfusion in patients with acute upper gastrointestinal bleeding to explore whether estimates for the participant- and cluster-average effects differed, to provide empirical evidence for whether informative cluster size may be present. For each outcome, we first estimated a participant-average effect using independence estimating equations, which are unbiased under informative cluster size. We then compared this to two further methods: (1) a cluster-average effect estimated using either weighted independence estimating equations or unweighted cluster-level summaries, and (2) estimates from a mixed-effects model or generalised estimating equations with an exchangeable correlation structure. We then performed a small simulation study to evaluate whether observed differences between cluster- and participant-average estimates were likely to occur even if no informative cluster size was present., Results: For most outcomes, treatment effect estimates from different methods were similar. However, differences of >10% occurred between participant- and cluster-average estimates for 5 of 17 outcomes (29%). We also observed several notable differences between estimates from mixed-effects models or generalised estimating equations with an exchangeable correlation structure and those based on independence estimating equations. For example, for the EQ-5D VAS score, the independence estimating equation estimate of the participant-average difference was 4.15 (95% confidence interval: -3.37 to 11.66), compared with 2.84 (95% confidence interval: -7.37 to 13.04) for the cluster-average independence estimating equation estimate, and 3.23 (95% confidence interval: -6.70 to 13.16) from a mixed-effects model. Similarly, for thromboembolic/ischaemic events, the independence estimating equation estimate for the participant-average odds ratio was 0.43 (95% confidence interval: 0.07 to 2.48), compared with 0.33 (95% confidence interval: 0.06 to 1.77) from the cluster-average estimator., Conclusion: In this re-analysis, we found that estimates from the various approaches could differ, which may be due to the presence of informative cluster size. Careful consideration of the estimand and the plausibility of assumptions underpinning each estimator can help ensure an appropriate analysis methods are used. Independence estimating equations and the analysis of cluster-level summaries (with appropriate weighting for each to correspond to either the participant-average or cluster-average treatment effect) are a desirable choice when informative cluster size is deemed possible, due to their unbiasedness in this setting., Competing Interests: Declaration of conflicting interestsThe author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Many trials use stratified randomisation, where participants are randomised within strata defined by one or more baseline covariates. While it is important to adjust for stratification variables in the analysis, the appropriate method of adjustment is unclear when stratification variables are affected by misclassification and hence some participants are randomised in the incorrect stratum. We conducted a simulation study to compare methods of adjusting for stratification variables affected by misclassification in the analysis of continuous outcomes when all or only some stratification errors are discovered, and when the treatment effect or treatment-by-covariate interaction effect is of interest. The data were analysed using linear regression with no adjustment, adjustment for the strata used to perform the randomisation (randomisation strata), adjustment for the strata if all errors are corrected (true strata), and adjustment for the strata after some errors are discovered and corrected (updated strata). The unadjusted model performed poorly in all settings. Adjusting for the true strata was optimal, while the relative performance of adjusting for the randomisation strata or the updated strata varied depending on the setting. As the true strata are unlikely to be known with certainty in practice, we recommend using the updated strata for adjustment and performing subgroup analyses, provided the discovery of errors is unlikely to depend on treatment group, as expected in blinded trials. Greater transparency is needed in the reporting of stratification errors and how they were addressed in the analysis., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Background: Clinical trials aim to draw conclusions about the effects of treatments, but a trial can address many different potential questions. For example, does the treatment work well for patients who take it as prescribed? Or does it work regardless of whether patients take it exactly as prescribed? Since different questions can lead to different conclusions on treatment benefit, it is important to clearly understand what treatment effect a trial aims to investigate-this is called the 'estimand'. Using estimands helps to ensure trials are designed and analysed to answer the questions of interest to different stakeholders, including patients and public. However, there is uncertainty about whether patients and public would like to be involved in defining estimands and how to do so. Public partners are patients and/or members of the public who are part of, or advise, the research team. We aimed to (i) co-develop a tool with public partners that helps explain what an estimand is and (ii) explore public partner's perspectives on the importance of discussing estimands during trial design., Methods: An online consultation meeting was held with 5 public partners of mixed age, gender and ethnicities, from various regions of the UK. Public partner opinions were collected and a practical tool describing estimands, drafted before the meeting by the research team, was developed. Afterwards, the tool was refined, and additional feedback sought via email., Results: Public partners want to be involved in estimand discussions. They found an introductory tool, to be presented and described to them by a researcher, helpful for starting a discussion about estimands in a trial design context. They recommended storytelling, analogies and visual aids within the tool. Four topics related to public partners' involvement in defining estimands were identified: (i) the importance of addressing questions that are relevant to patients and public in trials, (ii) involving public partners early on, (iii) a need for education and communication for all stakeholders and (iv) public partners and researchers working together., Conclusions: We co-developed a tool for researchers and public partners to use to facilitate the involvement of public partners in estimand discussions., (© 2023. The Author(s).)

Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This is because most authors do not attempt to describe what the treatment effect represents, and instead require readers to deduce this based on the reported statistical methods. However, this approach is challenging, because many methods provide counterintuitive results. For example, some methods include data from all patients, yet the resulting treatment effect applies only to a subset of patients, whereas other methods will exclude certain patients while results will apply to everyone. Additionally, some analyses provide estimates pertaining to hypothetical settings in which patients never die or discontinue treatment. Herein we introduce estimands as a solution to the aforementioned problem. An estimand is a clear description of what the treatment effect represents, thus saving readers the necessity of trying to infer this from study methods and potentially getting it wrong. We provide examples of how estimands can remove ambiguity from reported treatment effects and describe their current use in practice. The crux of our argument is that readers should not have to infer what investigators are estimating; they should be told explicitly., (© The Author(s) 2023. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health.)

Background: Poor retention in randomised trials can lead to serious consequences to their validity. Studies within trials (SWATs) are used to identify the most effective interventions to increase retention. Many interventions could be applied at any follow-up time point, but SWATs commonly assess interventions at a single time point, which can reduce efficiency., Methods: The re-randomisation design allows participants to be re-enrolled and re-randomised whenever a new retention opportunity occurs (i.e. a new follow-up time point where the intervention could be applied). The main advantages are as follows: (a) it allows the estimation of an average effect across time points, thus increasing generalisability; (b) it can be more efficient than a parallel arm trial due to increased sample size; and (c) it allows subgroup analyses to estimate effectiveness at different time points. We present a case study where the re-randomisation design is used in a SWAT., Results: In our case study, the host trial is a dental trial with two available follow-up points. The Sticker SWAT tests whether adding the trial logo's sticker to the questionnaire's envelope will result in a higher response rate compared with not adding the sticker. The primary outcome is the response rate to postal questionnaires. The re-randomisation design could double the available sample size compared to a parallel arm trial, resulting in the ability to detect an effect size around 28% smaller., Conclusion: The re-randomisation design can increase the efficiency and generalisability of SWATs for trials with multiple follow-up time points., (© 2023. The Author(s).)

Background: Cluster-randomized trials (CRTs) involve randomizing groups of individuals (e.g. hospitals, schools or villages) to different interventions. Various approaches exist for analysing CRTs but there has been little discussion around the treatment effects (estimands) targeted by each., Methods: We describe the different estimands that can be addressed through CRTs and demonstrate how choices between different analytic approaches can impact the interpretation of results by fundamentally changing the question being asked, or, equivalently, the target estimand., Results: CRTs can address either the participant-average treatment effect (the average treatment effect across participants) or the cluster-average treatment effect (the average treatment effect across clusters). These two estimands can differ when participant outcomes or the treatment effect depends on the cluster size (referred to as 'informative cluster size'), which can occur for reasons such as differences in staffing levels or types of participants between small and large clusters. Furthermore, common estimators, such as mixed-effects models or generalized estimating equations with an exchangeable working correlation structure, can produce biased estimates for both the participant-average and cluster-average treatment effects when cluster size is informative. We describe alternative estimators (independence estimating equations and cluster-level analyses) that are unbiased for CRTs even when informative cluster size is present., Conclusion: We conclude that careful specification of the estimand at the outset can ensure that the study question being addressed is clear and relevant, and, in turn, that the selected estimator provides an unbiased estimate of the desired quantity., (© The Author(s) 2022. Published by Oxford University Press on behalf of the International Epidemiological Association.)

Background/aims: Tuberculosis remains one of the leading causes of death from an infectious disease globally. Both choices of outcome definitions and approaches to handling events happening post-randomisation can change the treatment effect being estimated, but these are often inconsistently described, thus inhibiting clear interpretation and comparison across trials., Methods: Starting from the ICH E9(R1) addendum's definition of an estimand, we use our experience of conducting large Phase III tuberculosis treatment trials and our understanding of the estimand framework to identify the key decisions regarding how different event types are handled in the primary outcome definition, and the important points that should be considered in making such decisions. A key issue is the handling of intercurrent (i.e. post-randomisation) events (ICEs) which affect interpretation of or preclude measurement of the intended final outcome. We consider common ICEs including treatment changes and treatment extension, poor adherence to randomised treatment, re-infection with a new strain of tuberculosis which is different from the original infection, and death. We use two completed tuberculosis trials (REMoxTB and STREAM Stage 1) as illustrative examples. These trials tested non-inferiority of new tuberculosis treatment regimens versus a control regimen. The primary outcome was a binary composite endpoint, 'favourable' or 'unfavourable', which was constructed from several components., Results: We propose the following improvements in handling the above-mentioned ICEs and loss to follow-up (a post-randomisation event that is not in itself an ICE). First, changes to allocated regimens should not necessarily be viewed as an unfavourable outcome; from the patient perspective, the potential harms associated with a change in the regimen should instead be directly quantified. Second, handling poor adherence to randomised treatment using a per-protocol analysis does not necessarily target a clear estimand; instead, it would be desirable to develop ways to estimate the treatment effects more relevant to programmatic settings. Third, re-infection with a new strain of tuberculosis could be handled with different strategies, depending on whether the outcome of interest is the ability to attain culture negativity from infection with any strain of tuberculosis, or specifically the presenting strain of tuberculosis. Fourth, where possible, death could be separated into tuberculosis-related and non-tuberculosis-related and handled using appropriate strategies. Finally, although some losses to follow-up would result in early treatment discontinuation, patients lost to follow-up before the end of the trial should not always be classified as having an unfavourable outcome. Instead, loss to follow-up should be separated from not completing the treatment, which is an ICE and may be considered as an unfavourable outcome., Conclusion: The estimand framework clarifies many issues in tuberculosis trials but also challenges trialists to justify and improve their outcome definitions. Future trialists should consider all the above points in defining their outcomes.

Factorial trials offer an efficient method to evaluate multiple interventions in a single trial, however the use of additional treatments can obscure research objectives, leading to inappropriate analytical methods and interpretation of results. We define a set of estimands for factorial trials, and describe a framework for applying these estimands, with the aim of clarifying trial objectives and ensuring appropriate primary and sensitivity analyses are chosen. This framework is intended for use in factorial trials where the intent is to conduct "two-trials-in-one" (ie, to separately evaluate the effects of treatments A and B), and is comprised of four steps: (i) specifying how additional treatment(s) (eg, treatment B) will be handled in the estimand, and how intercurrent events affecting the additional treatment(s) will be handled; (ii) designating the appropriate factorial estimator as the primary analysis strategy; (iii) evaluating the interaction to assess the plausibility of the assumptions underpinning the factorial estimator; and (iv) performing a sensitivity analysis using an appropriate multiarm estimator to evaluate to what extent departures from the underlying assumption of no interaction may affect results. We show that adjustment for other factors is necessary for noncollapsible effect measures (such as odds ratio), and through a trial re-analysis we find that failure to consider the estimand could lead to inappropriate interpretation of results. We conclude that careful use of the estimands framework clarifies research objectives and reduces the risk of misinterpretation of trial results, and should become a standard part of both the protocol and reporting of factorial trials., (© 2022 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Background: Factorial designs and multi-arm multi-stage (MAMS) platform designs have many advantages, but the practical advantages and disadvantages of combining the two designs have not been explored., Methods: We propose practical methods for a combined design within the platform trial paradigm where some interventions are not expected to interact and could be given together., Results: We describe the combined design and suggest diagrams that can be used to represent it. Many properties are common both to standard factorial designs, including the need to consider interactions between interventions and the impact of intervention efficacy on power of other comparisons, and to standard multi-arm multi-stage designs, including the need to pre-specify procedures for starting and stopping intervention comparisons. We also identify some specific features of the factorial-MAMS design: timing of interim and final analyses should be determined by calendar time or total observed events; some non-factorial modifications may be useful; eligibility criteria should be broad enough to include any patient eligible for any part of the randomisation; stratified randomisation may conveniently be performed sequentially; and analysis requires special care to use only concurrent controls., Conclusion: A combined factorial-MAMS design can combine the efficiencies of factorial trials and multi-arm multi-stage platform trials. It allows us to address multiple research questions under one protocol and to test multiple new treatment options, which is particularly important when facing a new emergent infection such as COVID-19.

Precise specification of the research question and associated treatment effect of interest is essential in clinical research, yet recent work shows that they are often incompletely specified. The ICH E9 (R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials introduces a framework that supports researchers in precisely and transparently specifying the treatment effect they aim to estimate in their clinical trial. In this paper, we present practical examples to demonstrate to all researchers involved in clinical trials how estimands can help them to specify the research question, lead to a better understanding of the treatment effect to be estimated and hence increase the probability of success of the trial., Competing Interests: Competing interests: We have read and understood the BMJ Group policy on declaration of interests and declare the following interests: TPC and AP are employed by the clinical research organisation ICON., (© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY. Published by BMJ.)

Background: An estimand is a precise description of the treatment effect to be estimated from a trial (the question) and is distinct from the methods of statistical analysis (how the question is to be answered). The potential use of estimands to improve trial research and reporting has been underpinned by the recent publication of the ICH E9(R1) Addendum on the use of estimands in clinical trials in 2019. We set out to assess how well estimands are described in published trial protocols., Methods: We reviewed 50 trial protocols published in October 2020 in Trials and BMJ Open. For each protocol, we determined whether the estimand for the primary outcome was explicitly stated, not stated but inferable (i.e. could be constructed from the information given), or not inferable., Results: None of the 50 trials explicitly described the estimand for the primary outcome, and in 74% of trials, it was impossible to infer the estimand from the information included in the protocol. The population attribute of the estimand could not be inferred in 36% of trials, the treatment condition attribute in 20%, the population-level summary measure in 34%, and the handling of intercurrent events in 60% (the strategy for handling non-adherence was not inferable in 32% of protocols, and the strategy for handling mortality was not inferable in 80% of the protocols for which it was applicable). Conversely, the outcome attribute was stated for all trials. In 28% of trials, three or more of the five estimand attributes could not be inferred., Conclusions: The description of estimands in published trial protocols is poor, and in most trials, it is impossible to understand exactly what treatment effect is being estimated. Given the utility of estimands to improve clinical research and reporting, this urgently needs to change., (© 2021. The Author(s).)

Background and Objective: Prespecification of statistical methods in clinical trial protocols and statistical analysis plans can help to deter bias from p-hacking but is only effective if the prespecified approach is made available., Study Design and Setting: For 100 randomized trials published in 2018 and indexed in PubMed, we evaluated how often a prespecified statistical analysis approach for the trial's primary outcome was publicly available. For each trial with an available prespecified analysis, we compared this with the trial publication to identify whether there were unexplained discrepancies., Results: Only 12 of 100 trials (12%) had a publicly available prespecified analysis approach for their primary outcome; this document was dated before recruitment began for only two trials. Of the 12 trials with an available prespecified analysis approach, 11 (92%) had one or more unexplained discrepancies. Only 4 of 100 trials (4%) stated that the statistician was blinded until the SAP was signed off, and only 10 of 100 (10%) stated the statistician was blinded until the database was locked., Conclusion: For most published trials, there is insufficient information available to determine whether the results may be subject to p-hacking. Where information was available, there were often unexplained discrepancies between the prespecified and final analysis methods., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Introduction: In a five-arm randomized clinical trial (RCT) with stratified randomization across 54 sites, we encountered low primary outcome event proportions, resulting in multiple sites with zero events either overall or in one or more study arms. In this paper, we systematically evaluated different statistical methods of accounting for center in settings with low outcome event proportions., Methods: We conducted a simulation study and a reanalysis of a completed RCT to compare five popular methods of estimating an odds ratio for multicenter trials with stratified randomization by center: (i) no center adjustment, (ii) random intercept model, (iii) Mantel-Haenszel model, (iv) generalized estimating equation (GEE) with an exchangeable correlation structure, and (v) GEE with small sample correction (GEE-small sample correction). We varied the number of total participants (200, 500, 1000, 5000), number of centers (5, 50, 100), control group outcome percentage (2%, 5%, 10%), true odds ratio (1, > 1), intra-class correlation coefficient (ICC) (0.025, 0.075), and distribution of participants across the centers (balanced, skewed)., Results: Mantel-Haenszel methods generally performed poorly in terms of power and bias and led to the exclusion of participants from the analysis because some centers had no events. Failure to account for center in the analysis generally led to lower power and type I error rates than other methods, particularly with ICC = 0.075. GEE had an inflated type I error rate except in some settings with a large number of centers. GEE-small sample correction maintained the type I error rate at the nominal level but suffered from reduced power and convergence issues in some settings when the number of centers was small. Random intercept models generally performed well in most scenarios, except with a low event rate (i.e., 2% scenario) and small total sample size (n ≤ 500), when all methods had issues., Discussion: Random intercept models generally performed best across most scenarios. GEE-small sample correction performed well when the number of centers was large. We do not recommend the use of Mantel-Haenszel, GEE, or models that do not account for center. When the expected event rate is low, we suggest that the statistical analysis plan specify an alternative method in the case of non-convergence of the primary method.

Results from clinical trials can be susceptible to bias if investigators choose their analysis approach after seeing trial data, as this can allow them to perform multiple analyses and then choose the method that provides the most favourable result (commonly referred to as 'p-hacking'). Pre-specification of the planned analysis approach is essential to help reduce such bias, as it ensures analytical methods are chosen in advance of seeing the trial data. For this reason, guidelines such as SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and ICH-E9 (International Conference for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use) require the statistical methods for a trial's primary outcome be pre-specified in the trial protocol. However, pre-specification is only effective if done in a way that does not allow p-hacking. For example, investigators may pre-specify a certain statistical method such as multiple imputation, but give little detail on how it will be implemented. Because there are many different ways to perform multiple imputation, this approach to pre-specification is ineffective, as it still allows investigators to analyse the data in different ways before deciding on a final approach. In this article, we describe a five-point framework (the Pre-SPEC framework) for designing a pre-specified analysis approach that does not allow p-hacking. This framework was designed based on the principles in the SPIRIT and ICH-E9 guidelines and is intended to be used in conjunction with these guidelines to help investigators design the statistical analysis strategy for the trial's primary outcome in the trial protocol.

Background: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking., Methods: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a 'pandemic-free world' and 'world including a pandemic' are of interest., Results: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a 'pandemic-free world', participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the 'world including a pandemic', all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption - potentially incorporating a pandemic time-period indicator and participant infection status - or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses., Conclusions: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Objectives: Factorial designs can allow efficient evaluation of multiple treatments within a single trial. We evaluated the design, analysis, and reporting in a sample of factorial trials., Study Design and Setting: Review of 2 × 2 factorial trials evaluating health-related interventions and outcomes in humans. Using Medline, we identified articles published between January 2015 and March 2018. We randomly selected 100 articles for inclusion., Results: Most trials (78%) did not provide a rationale for using a factorial design. Only 63 trials (63%) assessed the interaction for the primary outcome, and 39/63 (62%) made a further assessment for at least one secondary outcome. 12/63 trials (19%) identified a significant interaction for the primary outcome and 16/39 trials (41%) for at least one secondary outcome. Inappropriate methods of analysis to protect against potential negative effects from interactions were common, with 18 trials (18%) choosing the analysis method based on a preliminary test for interaction, and 13% (n = 10/75) of those conducting a factorial analysis including an interaction term in the model., Conclusion: Reporting of factorial trials was often suboptimal, and assessment of interactions was poor. Investigators often used inappropriate methods of analysis to try to protect against adverse effects of interactions., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Objectives: To evaluate whether noninferiority trials are designed to adequately preserve the historical treatment effect of their active comparators., Study Design and Setting: We reviewed 162 noninferiority trials published in high-impact medical journals. We assessed whether trials were designed to ensure that interventions could only be declared noninferior if they preserved at least 50% of the active comparator's historical treatment effect., Results: Only 25 of 162 trials (15%) were designed so that interventions could only be declared noninferior if they preserved at least 50% of the active comparator's historical treatment effect. Most trials did not provide evidence that the active comparator was effective (n = 101), provided inadequate evidence (n = 18), or used a noninferiority margin that was too wide (n = 18). In a subset of 61 noninferiority trials which referenced a prior randomized trial or meta-analysis evaluating the active comparator, only 25 (41%) used a noninferiority margin small enough to preserve at least 50% of the active comparator's treatment effect. Overall, 14 of 162 noninferiority trials (9%) would have allowed the intervention to be declared noninferior even if it was worse than either placebo or another historical control., Conclusion: Most noninferiority trials published in major medical journals could allow erroneous declarations of noninferiority., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Background: Pre-specification of outcomes is an important tool to guard against outcome switching in clinical trials. However, if the outcome is not sufficiently clearly defined, then different definitions could be applied and analysed, with only the most favourable result reported., Methods: In order to assess the impact that differing outcome definitions could have on treatment effect estimates, we re-analysed data from TRIGGER, a cluster randomised trial comparing two red blood cell transfusion strategies for patients with acute upper gastrointestinal bleeding. We varied several aspects of the definition of further bleeding: (1) the criteria for what constitutes a further bleeding episode; (2) how further bleeding is assessed; and (3) the time-point at which further bleeding is measured., Results: There were marked discrepancies in the estimated odds ratios (OR) (range 0.23-0.94) and corresponding P values (range < 0.001-0.89) between different outcome definitions. At the extremes, differing outcome definitions led to markedly different conclusions; one definition led to very little evidence of a treatment effect (OR = 0.94, 95% confidence interval [CI] = 0.37-2.40, P = 0.89), while another led to very strong evidence of a treatment effect (OR = 0.23, 95% CI = 0.11-0.50, P < 0.001)., Conclusions: Outcomes should be pre-specified in sufficient detail to avoid differing definitions being analysed and only the most favourable result being reported., Trial Registration: Clinical Trials.gov, NCT02105532 . Registered on 7 April 2014.

Objective: Re-randomization trials allow patients to be re-enrolled for multiple treatment episodes. However, it remains uncertain to what extent re-randomization improves recruitment compared to parallel group designs or whether treatment estimates might be affected., Study Design and Setting: We evaluated trials included in a recent Cochrane review of granulocyte colony-stimulating factors for patients with febrile neutropenia. We assessed the recruitment benefits of re-randomization trials; compared treatment effect estimates between re-randomization and parallel group designs; and assessed whether re-randomization led to higher rates of non-compliance and loss to follow-up in subsequent episodes., Results: We included 14 trials (5 re-randomization and 9 parallel group). The re-randomization trials recruited a median of 25% (range 16-66%) more episodes on average than they would have under a parallel-group design. Treatment effect estimates were similar between re-randomization and parallel group trials across all outcomes, though confidence intervals were wide. The re-randomization trials in this review reported no loss to follow-up and low rates of non-compliance (median 1.7%, range 0-8.9%)., Conclusions: In the setting of febrile neutropenia, re-randomization increased recruitment while providing similar estimates of treatment effect to parallel group trials, with minimal loss to follow-up or non-compliance. It appears to be safe and efficient alternative to parallel group designs in this setting., (Copyright © 2018 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: Incorrect classification of outcomes in clinical trials can lead to biased estimates of treatment effect and reduced power. Ensuring appropriate adjudication methods to minimize outcome misclassification is therefore essential. While there are many reported adjudication approaches, there is little consensus over which approach is best., Methods: Under the assumption of non-differential assessment (i.e. that misclassification rates are the same in each treatment arm, as would typically be the case when outcome assessors are blinded), we use simulation and theoretical results to address four different questions about outcome adjudication: (a) How many assessors should be used? (b) When is it better to use onsite or central assessment? (c) Should central assessors adjudicate all outcomes, or only suspected events? (d) Should central assessment with multiple assessors be done independently or through group consensus?, Results: No one adjudication approach performs optimally in all settings. The optimal approach depends on the misclassification rates of site and central assessors, and the correlation between assessors. We found: (a) there will generally be little incremental benefit to using more than three assessors and, for outcomes with very high correlation between assessors, using one assessor is sufficient; (b) when choosing between site and central assessors, the assessor with the smallest misclassification rate should be chosen; when these rates are unknown, a combination of one site assessor and two central assessors will provide good results across a range of scenarios; (c) having central assessors adjudicate only suspected events will typically increase bias, and should be avoided, unless the threshold for sending outcomes for central assessment is extremely low; (d) central assessors can adjudicate either independently or in a group, and the preferred option should be dictated by whichever is expected to have the lowest misclassification rate., Conclusions: Outcome adjudication is of critical importance to ensure validity of trial results, although no one approach is optimal across all settings. Investigators should choose the best strategy based on the specific characteristics of their trial. Regardless of the adjudication strategy chosen, assessors should be qualified and receive appropriate training.

Objective: To determine whether modifying an outcome definition to remove subjective elements reduced bias in a trial that could not use blinded outcome assessment., Study Design and Setting: Reanalysis of an open-label trial comparing a restrictive vs. liberal transfusion strategy for gastrointestinal bleeding. The usual definition of the primary outcome, further bleeding, allows subjective clinical symptoms to be used alone for diagnosis, whereas the definition used in the trial required more objective confirmation by endoscopy. We compared treatment effect estimates for these two definitions., Results: Fewer subjective symptom-identified events were confirmed using more objective methods in the restrictive arm (18%) than in the liberal arm (56%), indicating differential assessment between arms. An analysis using all events (both subjective and more objective) led to an odds ratio of 0.83 (95% confidence interval [CI]: 0.50-1.37). When only events confirmed using more objective methods were included, the odds ratio was 0.50 (95% CI: 0.32-0.78). The ratio of the odds ratios was 1.66, indicating that including unconfirmed events in the definition biased the treatment effect upward by 66%., Conclusion: Modifying the outcome definition to exclude subjective elements substantially reduced bias. This may be a useful strategy for reducing bias in trials that cannot blind outcome assessment., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Background: Blinded outcome assessment is recommended in open-label trials to reduce bias, however it is not always feasible. It is therefore important to find other means of reducing bias in these scenarios., Methods: We describe two randomised trials where blinded outcome assessment was not possible, and discuss the strategies used to reduce the possibility of bias., Results: TRIGGER was an open-label cluster randomised trial whose primary outcome was further bleeding. Because of the cluster randomisation, all researchers in a hospital were aware of treatment allocation and so could not perform a blinded assessment. A blinded adjudication committee was also not feasible as it was impossible to compile relevant information to send to the committee in a blinded manner. Therefore, the definition of further bleeding was modified to exclude subjective aspects (such as whether symptoms like vomiting blood were severe enough to indicate the outcome had been met), leaving only objective aspects (the presence versus absence of active bleeding in the upper gastrointestinal tract confirmed by an internal examination).TAPPS was an open-label trial whose primary outcome was whether the patient was referred for a pleural drainage procedure. Allowing a blinded assessor to decide whether to refer the patient for a procedure was not feasible as many clinicians may be reluctant to enrol patients into the trial if they cannot be involved in their care during follow-up. Assessment by an adjudication committee was not possible, as the outcome either occurred or did not. Therefore, the decision pathway for procedure referral was modified. If a chest x-ray indicated that more than a third of the pleural space filled with fluid, the patient could be referred for a procedure; otherwise, the unblinded clinician was required to reach a consensus on referral with a blinded assessor. This process allowed the unblinded clinician to be involved in the patient's care, while reducing the potential for bias., Conclusions: When blinded outcome assessment is not possible, it may be useful to modify the outcome definition or method of assessment to reduce the risk of bias., Trial Registration Trigger: ISRCTN85757829. Registered 26 July 2012.TAPPS: ISRCTN47845793. Registered 28 May 2012.

Background: The cluster randomized crossover (CRXO) design is gaining popularity in trial settings where individual randomization or parallel group cluster randomization is not feasible or practical. In a CRXO trial, not only are clusters of individuals rather than individuals themselves randomized to trial arms, but also each cluster participates in each arm of the trial at least once in separate periods of time.We will review publications of clinical trials undertaken in humans that have used the CRXO design. The aim of this systematic review is to summarize, as reported: the motivations for using the CRXO design, the values of the CRXO design parameters, the justification and methodology for the sample size calculations and analyses, and the quality of reporting the CRXO design aspects., Methods/design: We will identify reports of CRXO trials by systematically searching MEDLINE, PubMed, Cochrane Methodology Register, EMBASE, and CINAHL Plus. In addition, we will search for methodological articles that describe the CRXO design and conduct citation searches to identify any further CRXO trials. The references of all eligible trials will also be searched. We will screen the identified abstracts, and retrieve and assess for inclusion the full text for any potentially relevant articles. Data will be extracted from the full text independently by two reviewers. Descriptive summary statistics will be presented for the extracted data., Discussion: This systematic review will inform both researchers addressing CRXO methodology and trialists considering implementing the design. The results will allow focused methodological research of the CRXO design, provide practical examples for researchers of how CRXO trials have been conducted, including any shortcomings, and highlight areas where reporting and conduct may be improved.

Background: Adjustment for prognostic covariates can lead to increased power in the analysis of randomized trials. However, adjusted analyses are not often performed in practice., Methods: We used simulation to examine the impact of covariate adjustment on 12 outcomes from 8 studies across a range of therapeutic areas. We assessed (1) how large an increase in power can be expected in practice; and (2) the impact of adjustment for covariates that are not prognostic., Results: Adjustment for known prognostic covariates led to large increases in power for most outcomes. When power was set to 80% based on an unadjusted analysis, covariate adjustment led to a median increase in power to 92.6% across the 12 outcomes (range 80.6 to 99.4%). Power was increased to over 85% for 8 of 12 outcomes, and to over 95% for 5 of 12 outcomes. Conversely, the largest decrease in power from adjustment for covariates that were not prognostic was from 80% to 78.5%., Conclusions: Adjustment for known prognostic covariates can lead to substantial increases in power, and should be routinely incorporated into the analysis of randomized trials. The potential benefits of adjusting for a small number of possibly prognostic covariates in trials with moderate or large sample sizes far outweigh the risks of doing so, and so should also be considered.

Background: Sensitivity analyses are an important tool for understanding the extent to which the results of randomised trials depend upon the assumptions of the analysis. There is currently no guidance governing the choice of sensitivity analyses., Discussion: We provide a principled approach to choosing sensitivity analyses through the consideration of the following questions: 1) Does the proposed sensitivity analysis address the same question as the primary analysis? 2) Is it possible for the proposed sensitivity analysis to return a different result to the primary analysis? 3) If the results do differ, is there any uncertainty as to which will be believed? Answering all of these questions in the affirmative will help researchers to identify relevant sensitivity analyses. Treating analyses as sensitivity analyses when one or more of the answers are negative can be misleading and confuse the interpretation of studies. The value of these questions is illustrated with several examples., Summary: By removing unreasonable analyses that might have been performed, these questions will lead to relevant sensitivity analyses, which help to assess the robustness of trial results.

Factorial trials are an efficient method of assessing multiple treatments in a single trial, saving both time and resources. However, they rely on the assumption of no interaction between treatment arms. Ignoring the possibility of an interaction in the analysis can lead to bias and potentially misleading conclusions. Therefore, it is often recommended that the size of the interaction be assessed during analysis. This approach can be formalised as a two-stage analysis; if the interaction test is not significant, a factorial analysis (where all patients receiving treatment A are compared with all not receiving A, and similarly for treatment B) is performed. If the interaction is significant, the analysis reverts to that of a four-arm trial (where each treatment combination is regarded as a separate treatment arm). We show that estimated treatment effects from the two-stage analysis can be biased, even in the absence of a true interaction. This occurs because the interaction estimate is highly correlated with treatment effect estimates from a four-arm analysis. Simulations show that bias can be severe (over 100% in some cases), leading to inflated type I error rates. Therefore, the two-stage analysis should not be used in factorial trials. A preferable approach may be to design multi-arm trials (i.e. four separate treatment groups) instead. This approach leads to straightforward interpretation of results, is unbiased regardless of the presence of an interaction, and allows investigators to ensure adequate power by basing sample size requirements on a four-arm analysis., (Copyright © 2013 John Wiley & Sons, Ltd.)

Background: Previous research has suggested an association between more liberal red blood cell (RBC) transfusion and greater risk of further bleeding and mortality following acute upper gastrointestinal bleeding (AUGIB)., Methods and Design: The Transfusion in Gastrointestinal Bleeding (TRIGGER) trial is a pragmatic cluster-randomised feasibility trial which aims to evaluate the feasibility of implementing a restrictive vs. liberal RBC transfusion policy for adult patients admitted to hospital with AUGIB in the UK. This trial will help to inform the design and methodology of a phase III trial. The protocol for TRIGGER has been published in Transfusion Medicine Reviews. Recruitment began in September 2012 and was completed in March 2013. This update presents the statistical analysis plan, detailing how analysis of the TRIGGER trial will be performed. It is hoped that prospective publication of the full statistical analysis plan will increase transparency and give readers a clear overview of how TRIGGER will be analysed., Trial Registration: ISRCTN85757829.

Acute upper gastrointestinal bleeding (AUGIB) is the commonest reason for hospitalization with hemorrhage in the UK and the leading indication for transfusion of red blood cells (RBCs). Observational studies suggest an association between more liberal RBC transfusion and adverse patient outcomes, and a recent randomised trial reported increased further bleeding and mortality with a liberal transfusion policy. TRIGGER (Transfusion in Gastrointestinal Bleeding) is a pragmatic, cluster randomized trial which aims to evaluate the feasibility and safety of implementing a restrictive versus liberal RBC transfusion policy in adult patients admitted with AUGIB. The trial will take place in 6 UK hospitals, and each centre will be randomly allocated to a transfusion policy. Clinicians throughout each hospital will manage all eligible patients according to the transfusion policy for the 6-month trial recruitment period. In the restrictive centers, patients become eligible for RBC transfusion when their hemoglobin is <8 g/dL. In the liberal centers patients become eligible for transfusion once their hemoglobin is <10 g/dL. All clinicians will have the discretion to transfuse outside of the policy but will be asked to document the reasons for doing so. Feasibility outcome measures include protocol adherence, recruitment rate, and evidence of selection bias. Clinical outcome measures include further bleeding, mortality, thromboembolic events, and infections. Quality of life will be measured using the EuroQol EQ-5D at day 28, and the costs associated with hospitalization for AUGIB in the UK will be estimated. Consent will be sought from participants or their representatives according to patient capacity for use of routine hospital data and day 28 follow up. The study has ethical approval for conduct in England and Scotland. Results will be analysed according to a pre-defined statistical analysis plan and disseminated in peer reviewed publications to relevant stakeholders. The results of this study will inform the feasibility and design of a phase III randomized trial., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Background: The objective of a non-inferiority trial is to determine whether a new or existing treatment is not less effective than another existing or current treatment by more than a pre-specified margin, Δ, usually with the requirement that the new treatment has some other advantage such as reduced cost or lower toxicity. A possible but unusual result in a non-inferiority trial is for the confidence interval for the treatment effect to lie between zero and Δ, implying that the new treatment is both inferior and non-inferior to the control. Such a result could occur in non-inferiority trials with large sample sizes or large non-inferiority margins. The possibility of this scenario occurring has implications for interim analyses. In standard superiority trials, stopping guidelines are often based on the p value obtained from testing whether treatments are equally effective. In non-inferiority trials, however, even if a new treatment is found to be inferior to the control at an interim analysis, there may still be a reasonable chance of demonstrating non-inferiority in the final analysis., Purpose: To explore the issues arising from trials where a simultaneously inferior and non-inferior result could occur and to describe appropriate methods for deciding whether such trials should be stopped for futility at an interim analysis., Methods: Conditional power is used to assess futility or the inability of the trial to show non-inferiority at the final analysis, by calculating the probability of demonstrating non-inferiority in the final analysis conditional on the observed results and upon assumptions on the future results of the trial. The Bullous Pemphigoid Steroids and Tetracyclines Study (BLISTER) trial is an example of a trial where a simultaneous inferior and non-inferior result could occur. A method for calculating conditional power for non-inferiority using simulations is described and applied at a hypothetical interim analysis of this trial., Results: Stopping guidelines for futility based on conditional power are shown to be better suited to non-inferiority trials than the typical methods used in superiority trials. Simulations are a straightforward and flexible way of calculating conditional power., Limitations: Calculating conditional power relies on assumptions about future treatment efficacy, and therefore, a number of different conditional power values can be obtained. Careful consideration should be given to which assumptions are most likely to be true. Additionally, when choosing a stopping guideline for futility, consideration needs to be given to avoid overinflating the type II error rate., Conclusions: Conditional power is an appropriate tool for defining stopping guidelines for futility in non-inferiority trials, particularly those with large non-inferiority margins.