Search

Background and Objectives: Conducting high-quality randomized clinical trials (RCTs) is challenging and resource intensive. Funders and academic investigators depend on limited financial resources and, therefore, need empirical data for optimal budget planning. However, current literature lacks detailed empirical data on resource use and costs of investigator-sponsored RCTs. The aim of this study is to systematically collect cost data from investigator-sponsored RCTs from Switzerland, Germany, and the United Kingdom (UK)., Methods: Principal investigators were asked to share their RCT cost and resource use data and enter it into an online case report form. We assessed cost patterns, cost drivers, and specific cost items, examined costs by study phase (planning-, conduct-, and finalization phase), compared planned with actual RCT costs, and explored differences in cost patterns across countries, medical fields, and intervention types., Results: We included 93 RCTs which were initiated in Switzerland (n = 53; including eight conducted in low- and lower middle-income countries), Germany (n = 22), and the UK (n = 18). The median total trial cost in our RCT sample was $645,824 [interquartile range (IQR), $269,846-$1,577,924]. The median proportion of the total costs spent for planning phase was 27.5% [IQR, 20.6%-39.7%], for conduct phase 57.3% [IQR, 44.4%-66.3%], and for finalization phase 12.7% [IQR, 8.5%-19.3%] with little variation across countries. The items that contributed most to the total costs were protocol writing (7.2%; IQR 3.8%-10.6%), data management (5.0%; IQR 2.2%-8.1%) and follow-up (4.5%; IQR 2.3%-8.4%). Of the 66 RCTs with an available original budget, 46 (69.7%) exceeded the budget by over 50%. Use of routinely collected data to assess primary outcomes was independently associated with lower per patient- and lower total trial costs., Conclusion: Over a quarter of total trial costs were incurred in the planning phase, which is typically not fully funded. Two-thirds of RCTs exceeded their budget by more than 50%. Investigators and funders should consider empirical cost data to improve budgeting and funding practices., Competing Interests: Declaration of competing interest B.S. and M.B. report an unrestricted grant from Moderna for a study outside of the submitted work. There are no competing interests for any other author., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: Trials within Cohorts (TwiCs) is a pragmatic design approach that may overcome frequent challenges of traditional randomized trials such as slow recruitment, burdensome consent procedures, or limited external validity. This scoping review aims to identify all randomized controlled trials using the TwiCs design and to summarize their design characteristics, ways to obtain informed consent, output, reported challenges and mitigation strategies., Study Design and Setting: Systematic search of Medline, Embase, Cochrane, trial registries and citation tracking up to December 2022. TwiCs were defined as randomized trials embedded in a cohort with postrandomization consent for the intervention group and no specific postrandomization consent for the usual care control group. Information from identified TwiCs was extracted in duplicate from protocols, publications, and registry entries. We analyzed the information descriptively and qualitatively to highlight methodological challenges and solutions related to nonuptake of interventions and informed consent procedure., Results: We identified a total of 46 TwiCs conducted between 2005 and 2022 in 14 different countries by a handful of research groups. The most common medical fields were oncology (11/46; 24%), infectious diseases (8/46; 17%), and mental health (7/46; 15%). A typical TwiCs was investigator-initiated (46/46; 100%), publicly funded (36/46; 78%), and recruited outpatients (27/46; 59%). Excluding eight pilot trials, only 16/38 (42%) TwiCs adjusted their calculated sample size for nonuptake of the intervention, anticipating a median nonuptake of 25% (interquartile range 10%-32%) in the experimental arm. Seventeen TwiCs (45%) planned analyses to adjust effect estimates for nonuptake. Regarding informed consent, we observed three patterns: 1) three separate consents for cohort participation, randomization, and intervention (17/46; 37%); 2) combined consent for cohort participation and randomization and a separate intervention consent (10/46; 22%); and 3) consent only for cohort participation and intervention (randomization consent not mentioned; 19/46; 41%)., Conclusion: Existing TwiCs are globally scattered across a few research groups covering a wide range of medical fields and interventions. Despite the potential advantages, the number of TwiCs remains small. The variability in consent procedures and the possibility of substantial nonuptake of the intervention warrants further research to guide the planning, implementation, and analysis of TwiCs., Competing Interests: Declaration of competing interest Several coauthors (AA, CMS, HMV, CR, FC, FG, NDL, MB) are using the TwiCs design or are part of the academic TwiCs global network (https://www.twics.global). We declare no financial conflict of interest. There are no competing interests for any other author., (Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)

Competing Interests: Declaration of competing interest None declared.

Objectives: To update previous Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance by addressing inconsistencies and interpreting subgroup analyses., Study Design and Setting: Using an iterative process, we consulted with members of the GRADE working group through multiple rounds of written feedback and discussions at GRADE working group meetings., Results: The guidance complements previous guidance with clarification in two areas: (1) assessing inconsistency and (2) assessing the credibility of possible effect modifiers that might explain inconsistency. Specifically, the guidance clarifies that inconsistency refers to variability in results, not in study characteristics; that inconsistency assessment for binary outcomes requires consideration of both relative and absolute effects; how to decide between narrower and broader questions in systematic reviews and guidelines; that, with the same evidence, ratings of inconsistency may differ depending on the target of certainty rating; and how GRADE inconsistency ratings relate to a statistical measure of inconsistency I 2 depending on the context in which one views results. The second part of the guidance illustrates, based on a worked example, the use of the instrument to assess the credibility of effect modification analyses. The guidance explains the stepwise process of moving from a subgroup analysis to assessing the credibility of effect modification and, if found credible, to subgroup-specific effect estimates and GRADE certainty ratings., Conclusion: This updated guidance addresses specific conceptual and practical issues that systematic review authors frequently face when considering the degree of inconsistency in estimates of treatment effects across studies., (Copyright © 2023 The Author(s). Published by Elsevier Inc. All rights reserved.)

Background and Objectives: To evaluate reporting of minimal important difference (MID) estimates using anchor-based methods for patient-reported outcome measures (PROMs), and the association with reporting deficiencies on their credibility., Methods: Systematic survey of primary studies empirically estimating MIDs. We searched Medline, EMBASE, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database until October 2018. We evaluated study reporting, focusing on participants' demographics, intervention(s), characteristics of PROMs and anchors, and MID estimation method(s). We assessed the impact of reporting issues on credibility of MID estimates., Results: In 585 studies reporting on 5,324 MID estimates for 526 distinct PROMs, authors frequently failed to adequately report key characteristics of PROMs and MIDs, including minimum and maximum values of PROM scale, measure of variability accompanying the MID estimate and number of participants included in the MID calculation. Across MID estimates (n = 5,324), the most serious reporting issues impacting credibility included infrequent reporting of the correlation between the anchor and PROM (66%), inadequate details to judge precision of MID point estimate (13%), and insufficient information about the threshold used to ascertain MIDs (16%)., Conclusion: Serious issues of incomplete reporting in the MID literature threaten the optimal use of MID estimates to inform the magnitude of effects of interventions on PROMs., (Copyright © 2022 Elsevier Inc. All rights reserved.)

Background: To assess the current practice of developing and presenting methods guidance and explore opportunities for improvement., Study Design and Setting: We systematically surveyed methods guidance published in high-impact general and methodology-focused medical journals indexed in MEDLINE in 2020. We included articles that explicitly stated the objective to provide methods guidance for health research. We extracted characteristics related to findability, methods used for development, presentation, and transparency., Results: We included 105 methods guidance articles published in 12 different journals. Less than half had a structured abstract (42%) or was indexed with medical subject headings (38%) or author keywords (17%) related to guidance. Methods for development, reported in 42%, differed between reporting guidelines (n = 13, 100% reported methods) and other guidance articles (n = 92, 34% reported methods). Frequent methods for presentation were illustrative case studies (45%), research checklists (34%), and step-by-step guides (10%). Most articles did not describe the authors' expertise (22%). Conflicts of interest, reported in 34%, were often unclear., Conclusion: Potential areas for improving methods guidance include better findability through more consistent labeling and indexing and standards for development and reporting., (Copyright © 2022 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: To investigate the adherence of randomised controlled trial (RCT) protocols evaluating non-regulated interventions (including dietary interventions, surgical procedures, behavioural and lifestyle interventions, and exercise programmes) in comparison with regulated interventions to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 Statement., Methods: We conducted a repeated cross-sectional investigation in a random sample of RCT protocols approved in 2012 (n = 257) or 2016 (n = 292) by research ethics committees in Switzerland, Germany, or Canada. We investigated the proportion of accurately reported SPIRIT checklist items in protocols of trials with non-regulated as compared to regulated interventions., Results: Overall, 131 (24%) of trial protocols tested non-regulated interventions. In 2012, the median proportion of SPIRIT items reported in these protocols (59%, interquartile range [IQR], 53%-69%) was lower than in protocols with regulated interventions (median, 74%, IQR, 66%-80%). In 2016, the reporting quality of protocols with non-regulated interventions (median, 75%, IQR, 62%-83%) improved to the level of regulated intervention protocols, which had not changed on average., Conclusions: Reporting of RCT protocols evaluating non-regulated interventions improved between 2012 and 2016, although remained suboptimal. SPIRIT recommendations need to be further endorsed by researchers, ethics committees, funding agencies, and journals to optimize reporting of RCT protocols., (Copyright © 2021 The Author(s). Published by Elsevier Inc. All rights reserved.)

Objectives: The objective of the study was to develop an inventory summarizing all anchor-based minimal important difference (MID) estimates for patient-reported outcome measures (PROMs) available in the medical literature., Study Design and Setting: We searched MEDLINE, EMBASE, CINAHL, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database internal library (January 1989-October 2018). We included primary studies empirically calculating an anchor-based MID estimate for any PROM in adults and adolescents. Pairs of reviewers independently screened and selected studies, extracted data, and evaluated the credibility of the MIDs., Results: We identified 585 eligible studies, the majority conducted in Europe (n = 211) and North America (n = 179), reporting 5,324 MID estimates for 526 distinct PROMs. Investigators conducted their studies in the context of patients receiving surgical (n = 105, 18%), pharmacological (n = 85, 15%), rehabilitation (n = 65, 11%), or a combination of interventions (n = 194, 33%). Of all MID estimates, 59% (n = 3,131) used a global rating of change anchor. Major credibility limitations included weak correlation (n = 1,246, 23%) or no information regarding the correlation (n = 3,498, 66%) between the PROM and anchor and imprecision in the MID estimate (n = 2,513, 47%)., Conclusion: A large number of MIDs for assisting in the interpretation of PROMs exist. The MID inventory will facilitate the use of MID estimates to inform the interpretation of the magnitude of treatment effects in clinical research and guideline development., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Objective: The objective of this study was to provide guidance on the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to determine certainty in estimates of association between prognostic factors and future outcomes., Study Design and Setting: We developed our guidance through an iterative process that involved review of published systematic reviews and meta-analyses of prognostic factors, consultation with members, feedback, presentation, and discussion at the GRADE Working Group meetings., Results: For questions of prognosis, a body of observational evidence (potentially including patients enrolled in randomized controlled trials) begins as high certainty in the evidence. The five domains of GRADE for rating down certainty in the evidence, that is, risk of bias, imprecision, inconsistency, indirectness, and publication bias, as well as the domains for rating up, also apply to estimates of associations between prognostic factors and outcomes. One should determine if their ratings do not consider (noncontextualized) or consider (contextualized) the clinical context as this will may result in variable judgments on certainty of the evidence., Conclusions: The same principles GRADE proposed for bodies of evidence addressing treatment and overall prognosis work well in assessing individual prognostic factors, both in noncontextualized and contextualized settings., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Objective: The objective of the study was to systematically survey the methodological literature and collect suggested criteria for assessing the credibility of effect modification and associated rationales., Study Design and Setting: We searched MEDLINE, Embase, and WorldCat up to March 2018 for publications providing guidance for assessing the credibility of effect modification identified in randomized trials or meta-analyses. Teams of two investigators independently identified eligible publications and extracted credibility criteria and authors' rationale, reaching consensus through discussion. We created a taxonomy of criteria that we iteratively refined during data abstraction., Results: We identified 150 eligible publications that provided 36 criteria and associated rationales. Frequent criteria included significant test for interaction (n = 54), a priori hypothesis (n = 49), providing a causal explanation (n = 47), accounting for multiplicity (n = 45), testing a small number of effect modifiers (n = 38), and prespecification of analytic details (n = 39). For some criteria, we found more than one rationale; some criteria were connected through a common rationale. For some criteria, experts disagreed regarding their suitability (e.g., added value of stratified randomization; trustworthiness of biologic rationales)., Conclusion: Methodologists have expended substantial intellectual energy providing criteria for critical appraisal of apparent effect modification. Our survey highlights popular criteria, expert agreement and disagreement, and where more work is needed, including testing criteria in practice., (Copyright © 2019. Published by Elsevier Inc.)

Objective: We explored how investigators of ongoing or planned trials respond to the publication of a trial stopped early for benefit addressing a similar question., Study Design and Setting: We searched multiple databases from the date of publication of the truncated trial through August, 2015. Independent reviewers selected trials and extracted data., Results: We identified 207 trials truncated for early benefit; of which 102 (49%) were followed by subsequent trials (262 subsequent trials, median 2 per truncated trial, range 1-13). Only 99 (38%) provided a rationale justifying conducting a trial despite prior stopping. The top reasons were to address different population or setting (33%), skepticism of truncated trials findings because of small sample size (12%), inconsistency with other evidence (11%), or increased risk of bias (7%). We did not identify significant associations between subsequent trials and characteristics of truncated ones (risk of bias, precision, funding, or rigor of stopping decision)., Conclusion: About half of the trials stopped early for benefit were followed by subsequent trials addressing a similar question. This suggests that future trialists may have been skeptic about the decision to stop prior trials. A more rigorous threshold for stopping early for benefit is needed., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Background: One quarter of randomized clinical trials (RCTs) are prematurely discontinued and frequently remain unpublished. Trial registries can document whether a trial is ongoing, suspended, discontinued, or completed and therefore represent an important source for trial status information. The accuracy of this information is unclear., Objective: To examine the accuracy of completion status and reasons for discontinuation documented in trial registries as compared to corresponding publications of discontinued RCTs and to investigate potential predictors for accurate trial status information in registries., Methods: We conducted a cross-sectional study comparing information provided in publications (reference standard) to corresponding registry entries. First, we reviewed publications of RCTs providing information on both discontinuation and registration. We identified eligible publications through systematic searches of MEDLINE and EMBASE (2010-2014) and an international cohort of 1,017 RCTs initiated between 2000 and 2003. Second, pairs of investigators independently and in duplicate extracted data from publications and corresponding registry records. Third, for each discontinued RCT, we compared publication information to registry information. We used multivariable regression to examine whether accurate labeling of trials as discontinued (vs. other status) in the registry was associated with recent initiation of RCT, industry sponsorship, multicenter design, or larger sample size., Results: We identified 173 publications of RCTs that were discontinued due to slow recruitment (55%), harm (16%), futility (11%), benefit (5%), other reasons (3%), or multiple reasons (9%). Trials were registered with clinicaltrials.gov (77%), isrctn.com (14%), or other registries (8%). Of the 173 corresponding registry records, 77 (45%) trials were labeled as discontinued and 57 (33%) provided a reason for discontinuation (of which 53, 93%, provided the same reason as in the publication). Labeling of discontinued trials as discontinued (vs. other label) in corresponding trial registry records improved over time (adjusted odds ratio 1.16 per year, confidence interval 1.04-1.30) and was possibly associated with industry sponsorship (2.01, 0.99-4.07) but unlikely with multicenter status (0.81, 0.32-2.04) or sample size (1.07, 0.89-1.29)., Conclusions: Less than half of published discontinued RCTs were accurately labelled as discontinued in corresponding registry records. One-third of registry records provided a reason for discontinuation. Current trial status information in registries should be viewed with caution., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Objective: To collect and classify reported reasons for recruitment failure in discontinued randomized controlled trials (RCTs) and to assess reporting quality., Methods: We systematically searched MEDLINE and EMBASE (2010-2014) and a previous cohort of RCTs for published RCTs reporting trial discontinuation due to poor recruitment. Teams of two investigators selected eligible RCTs working independently and extracted information using standardized forms. We used an iterative approach to classify reasons for poor recruitment., Results: We included 172 RCTs discontinued due to poor recruitment (including 26 conference abstracts and 63 industry-funded RCTs). Of those, 131 (76%) reported one or more reasons for discontinuation due to poor recruitment. We identified 28 different reasons for recruitment failure; most frequently mentioned were overestimation of prevalence of eligible participants and prejudiced views of recruiters and participants on trial interventions. Few RCTs reported relevant details about the recruitment process such as how eligible participants were identified, the number of patients assessed for eligibility, and who actually recruited participants., Conclusion: Our classification could serve as a checklist to assist investigators in the planning of RCTs. Most reasons for recruitment failure seem preventable with a pilot study that applies the planned informed consent procedure., (Copyright © 2016 Elsevier Inc. All rights reserved.)