Search

Background: It is unclear how multiple treatment comparisons are managed in the analysis of multi-arm trials, particularly related to reducing type I (false positive) and type II errors (false negative)., Methods: We conducted a cohort study of clinical-trial protocols that were approved by research ethics committees in the UK, Switzerland, Germany and Canada in 2012. We examined the use of multiple-testing procedures to control the overall type I error rate. We created a decision tool to determine the need for multiple-testing procedures. We compared the result of the decision tool to the analysis plan in the protocol. We also compared the pre-specified analysis plans in trial protocols to their publications., Results: Sixty-four protocols for multi-arm trials were identified, of which 50 involved multiple testing. Nine of 50 trials (18%) used a single-step multiple-testing procedures such as a Bonferroni correction and 17 (38%) used an ordered sequence of primary comparisons to control the overall type I error. Based on our decision tool, 45 of 50 protocols (90%) required use of a multiple-testing procedure but only 28 of the 45 (62%) accounted for multiplicity in their analysis or provided a rationale if no multiple-testing procedure was used. We identified 32 protocol-publication pairs, of which 8 planned a global-comparison test and 20 planned a multiple-testing procedure in their trial protocol. However, four of these eight trials (50%) did not use the global-comparison test. Likewise, 3 of the 20 trials (15%) did not perform the multiple-testing procedure in the publication. The sample size of our study was small and we did not have access to statistical-analysis plans for the included trials in our study., Conclusions: Strategies to reduce type I and type II errors are inconsistently employed in multi-arm trials. Important analytical differences exist between planned analyses in clinical-trial protocols and subsequent publications, which may suggest selective reporting of analyses., (© The Author(s) 2020; all rights reserved. Published by Oxford University Press on behalf of the International Epidemiological Association.)

Objectives: Factorial designs can allow efficient evaluation of multiple treatments within a single trial. We evaluated the design, analysis, and reporting in a sample of factorial trials., Study Design and Setting: Review of 2 × 2 factorial trials evaluating health-related interventions and outcomes in humans. Using Medline, we identified articles published between January 2015 and March 2018. We randomly selected 100 articles for inclusion., Results: Most trials (78%) did not provide a rationale for using a factorial design. Only 63 trials (63%) assessed the interaction for the primary outcome, and 39/63 (62%) made a further assessment for at least one secondary outcome. 12/63 trials (19%) identified a significant interaction for the primary outcome and 16/39 trials (41%) for at least one secondary outcome. Inappropriate methods of analysis to protect against potential negative effects from interactions were common, with 18 trials (18%) choosing the analysis method based on a preliminary test for interaction, and 13% (n = 10/75) of those conducting a factorial analysis including an interaction term in the model., Conclusion: Reporting of factorial trials was often suboptimal, and assessment of interactions was poor. Investigators often used inappropriate methods of analysis to try to protect against adverse effects of interactions., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Importance: While guidance on statistical principles for clinical trials exists, there is an absence of guidance covering the required content of statistical analysis plans (SAPs) to support transparency and reproducibility., Objective: To develop recommendations for a minimum set of items that should be addressed in SAPs for clinical trials, developed with input from statisticians, previous guideline authors, journal editors, regulators, and funders., Design: Funders and regulators (n = 39) of randomized trials were contacted and the literature was searched to identify existing guidance; a survey of current practice was conducted across the network of UK Clinical Research Collaboration-registered trial units (n = 46, 1 unit had 2 responders) and a Delphi survey (n = 73 invited participants) was conducted to establish consensus on SAPs. The Delphi survey was sent to statisticians in trial units who completed the survey of current practice (n = 46), CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guideline authors (n = 16), pharmaceutical industry statisticians (n = 3), journal editors (n = 9), and regulators (n = 2) (3 participants were included in 2 groups each), culminating in a consensus meeting attended by experts (N = 12) with representatives from each group. The guidance subsequently underwent critical review by statisticians from the surveyed trial units and members of the expert panel of the consensus meeting (N = 51), followed by piloting of the guidance document in the SAPs of 5 trials., Findings: No existing guidance was identified. The registered trials unit survey (46 responses) highlighted diversity in current practice and confirmed support for developing guidance. The Delphi survey (54 of 73, 74% participants completing both rounds) reached consensus on 42% (n = 46) of 110 items. The expert panel (N = 12) agreed that 63 items should be included in the guidance, with an additional 17 items identified as important but may be referenced elsewhere. Following critical review and piloting, some overlapping items were combined, leaving 55 items., Conclusions and Relevance: Recommendations are provided for a minimum set of items that should be addressed and included in SAPs for clinical trials. Trial registration, protocols, and statistical analysis plans are critically important in ensuring appropriate reporting of clinical trials.

Competing Interests: Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following interests: TH, PG, DM, DA, and RP are members of the team that developed the TIDieR guide. DM led development of PRISMA and PRISMA-P. DA, DM, PR, and PG are directors of the EQUATOR Centres in Oxford, Ottawa, France, and Australia, respectively.

The selection of appropriate outcomes is crucial when designing clinical trials in order to compare the effects of different interventions directly. For the findings to influence policy and practice, the outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. It is now widely acknowledged that insufficient attention has been paid to the choice of outcomes measured in clinical trials. Researchers are increasingly addressing this issue through the development and use of a core outcome set, an agreed standardised collection of outcomes which should be measured and reported, as a minimum, in all trials for a specific clinical area.Accumulating work in this area has identified the need for guidance on the development, implementation, evaluation and updating of core outcome sets. This Handbook, developed by the COMET Initiative, brings together current thinking and methodological research regarding those issues. We recommend a four-step process to develop a core outcome set. The aim is to update the contents of the Handbook as further research is identified.

N-of-1 trials provide a mechanism for making evidence-based treatment decisions for an individual patient. They use key methodological elements of group clinical trials to evaluate treatment effectiveness in a single patient, for situations that cannot always accommodate large-scale trials: rare diseases, comorbid conditions, or in patients using concurrent therapies. Improvement in the reporting and clarity of methods and findings in N-of-1 trials is essential for reader to gauge the validity of trials and to replicate successful findings. A Consolidated Standards of Reporting Trials (CONSORT) extension for N-of-1 trials (CENT 2015) provides guidance on the reporting of individual and series of N-of-1 trials. CENT provides additional guidance for 14 of the 25 items of the CONSORT 2010 checklist, recommends a diagram for depicting an individual N-of-1 trial, and modifies the CONSORT flow diagram to address the flow of a series of N-of-1 trials. The rationale, development process, and CENT 2015 checklist and diagrams are reported in this document., (Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.)

N-of-1 trials are a useful tool for clinicians who want to determine the effectiveness of a treatment in a particular individual. The reporting of N-of-1 trials has been variable and incomplete, hindering their usefulness in clinical decision making and by future researchers. This document presents the CONSORT (Consolidated Standards of Reporting Trials) extension for N-of-1 trials (CENT 2015). CENT 2015 extends the CONSORT 2010 guidance to facilitate the preparation and appraisal of reports of an individual N-of-1 trial or a series of prospectively planned, multiple, crossover N-of-1 trials. CENT 2015 elaborates on 14 items of the CONSORT 2010 checklist, totalling 25 checklist items (44 sub-items), and recommends diagrams to help authors document the progress of one participant through a trial or more than one participant through a trial or series of trials, as applicable. Examples of good reporting and evidence based rationale for CENT 2015 checklist items are provided., (Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.)

Background: The CONsolidated Standards Of Reporting Trials (CONSORT) Statement provides a minimum standard set of items to be reported in published clinical trials; it has received widespread recognition within the biomedical publishing community. This research aims to provide an update on the endorsement of CONSORT by high impact medical journals., Methods: We performed a cross-sectional examination of the online "Instructions to Authors" of 168 high impact factor (2012) biomedical journals between July and December 2014. We assessed whether the text of the "Instructions to Authors" mentioned the CONSORT Statement and any CONSORT extensions, and we quantified the extent and nature of the journals' endorsements of these. These data were described by frequencies. We also determined whether journals mentioned trial registration and the International Committee of Medical Journal Editors (ICMJE; other than in regards to trial registration) and whether either of these was associated with CONSORT endorsement (relative risk and 95 % confidence interval). We compared our findings to the two previous iterations of this survey (in 2003 and 2007). We also identified the publishers of the included journals., Results: Sixty-three percent (106/168) of the included journals mentioned CONSORT in their "Instructions to Authors." Forty-four endorsers (42 %) explicitly stated that authors "must" use CONSORT to prepare their trial manuscript, 38 % required an accompanying completed CONSORT checklist as a condition of submission, and 39 % explicitly requested the inclusion of a flow diagram with the submission. CONSORT extensions were endorsed by very few journals. One hundred and thirty journals (77 %) mentioned ICMJE, and 106 (63 %) mentioned trial registration., Conclusions: The endorsement of CONSORT by high impact journals has increased over time; however, specific instructions on how CONSORT should be used by authors are inconsistent across journals and publishers. Publishers and journals should encourage authors to use CONSORT and set clear expectations for authors about compliance with CONSORT.

Objective: The aim of this study is to review highly cited articles that focus on non-publication of studies, and to develop a consistent and comprehensive approach to defining (non-) dissemination of research findings., Setting: We performed a scoping review of definitions of the term 'publication bias' in highly cited publications., Participants: Ideas and experiences of a core group of authors were collected in a draft document, which was complemented by the findings from our literature search., Interventions: The draft document including findings from the literature search was circulated to an international group of experts and revised until no additional ideas emerged and consensus was reached., Primary Outcomes: We propose a new approach to the comprehensive conceptualisation of (non-) dissemination of research., Secondary Outcomes: Our 'What, Who and Why?' approach includes issues that need to be considered when disseminating research findings (What?), the different players who should assume responsibility during the various stages of conducting a clinical trial and disseminating clinical trial documents (Who?), and motivations that might lead the various players to disseminate findings selectively, thereby introducing bias in the dissemination process (Why?)., Conclusions: Our comprehensive framework of (non-) dissemination of research findings, based on the results of a scoping literature search and expert consensus will facilitate the development of future policies and guidelines regarding the multifaceted issue of selective publication, historically referred to as 'publication bias'., (Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/)

Background: The independent oversight of clinical trials, which is recommended by the Medical Research Council (MRC) Guidelines for Good Clinical Practice, is typically provided by an independent advisory Data Monitoring Committee (DMC) and an independent executive committee, to whom the DMC makes recommendations. The detailed roles and function of this executive committee, known as the Trial Steering Committee (TSC), have not previously been studied or reviewed since those originally proposed by the MRC in 1998., Methods: An expert panel (n = 7) was convened comprising statisticians, clinicians and trial methodologists with prior TSC experience. Twelve questions about the role and responsibilities of the TSC were discussed by the panel at two full-day meetings. Each meeting was transcribed in full and the discussions were summarised., Results: The expert panel reached agreement on the role of the TSC, to which it was accountable, the membership, the definition of independence, and the experience and training needed. The management of ethical issues, difficult/complex situations and issues the TSC should not ask the DMC to make recommendations on were more difficult to discuss without specific examples, but support existed for further work to help share issues and to provide appropriate training for TSC members. Additional topics discussed, which had not been identified by previous work relating to the DMCs but were pertinent to the role of the TSC, included the following: review of data sharing requests, indemnity, lifespan of the TSC, general TSC administration, and the roles of both the Funder and the Sponsor., Conclusions: This paper presents recommendations that will contribute to the revision and update of the MRC TSC terms of reference. Uncertainty remains in some areas due to the absence of real-life examples; future guidance on these issues would benefit from a repository of case studies. Notably, the role of a patient and public involvement (PPI) contributor was not discussed, and further work is warranted to explore the role of a PPI contributor in independent trial oversight.

The protocol of a clinical trial serves as the foundation for study planning, conduct, reporting, and appraisal. However, trial protocols and existing protocol guidelines vary greatly in content and quality. This article describes the systematic development and scope of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013, a guideline for the minimum content of a clinical trial protocol. The 33-item SPIRIT checklist applies to protocols for all clinical trials and focuses on content rather than format. The checklist recommends a full description of what is planned; it does not prescribe how to design or conduct a trial. By providing guidance for key content, the SPIRIT recommendations aim to facilitate the drafting of high-quality protocols. Adherence to SPIRIT would also enhance the transparency and completeness of trial protocols for the benefit of investigators, trial participants, patients, sponsors, funders, research ethics committees or institutional review boards, peer reviewers, journals, trial registries, policymakers, regulators, and other key stakeholders., Competing Interests: Conflictos de interés posibles. las declaraciones pueden verse en www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M12-1905

I reflect on the evolution of meta-analysis in clinical research over 35 years since I first encountered the key ideas in the late 1970s., (Copyright © 2015 Douglas G. Altman.)

Background: Core outcome sets can increase the efficiency and value of research and, as a result, there are an increasing number of studies looking to develop core outcome sets (COS). However, the credibility of a COS depends on both the use of sound methodology in its development and clear and transparent reporting of the processes adopted. To date there is no reporting guideline for reporting COS studies. The aim of this programme of research is to develop a reporting guideline for studies developing COS and to highlight some of the important methodological considerations in the process., Methods/design: The study will include a reporting guideline item generation stage which will then be used in a Delphi study. The Delphi study is anticipated to include two rounds. The first round will ask stakeholders to score the items listed and to add any new items they think are relevant. In the second round of the process, participants will be shown the distribution of scores for all stakeholder groups separately and asked to re-score. A final consensus meeting will be held with an expert panel and stakeholder representatives to review the guideline item list. Following the consensus meeting, a reporting guideline will be drafted and review and testing will be undertaken until the guideline is finalised. The final outcome will be the COS-STAR (Core Outcome Set-STAndards for Reporting) guideline for studies developing COS and a supporting explanatory document., Discussion: To assess the credibility and usefulness of a COS, readers of a COS development report need complete, clear and transparent information on its methodology and proposed core set of outcomes. The COS-STAR guideline will potentially benefit all stakeholders in COS development: COS developers, COS users, e.g. trialists and systematic reviewers, journal editors, policy-makers and patient groups.

Objective: To assess the waste of research related to inadequate methods in trials included in Cochrane reviews and to examine to what extent this waste could be avoided. A secondary objective was to perform a simulation study to re-estimate this avoidable waste if all trials were adequately reported., Design: Methodological review and simulation study., Data Sources: Trials included in the meta-analysis of the primary outcome of Cochrane reviews published between April 2012 and March 2013., Data Extraction and Synthesis: We collected the risk of bias assessment made by the review authors for each trial. For a random sample of 200 trials with at least one domain at high risk of bias, we re-assessed risk of bias and identified all related methodological problems. For each problem, possible adjustments were proposed that were then validated by an expert panel also evaluating their feasibility (easy or not) and cost. Avoidable waste was defined as trials with at least one domain at high risk of bias for which easy adjustments with no or minor cost could change all domains to low risk. In the simulation study, after extrapolating our re-assessment of risk of bias to all trials, we considered each domain rated as unclear risk of bias as missing data and used multiple imputations to determine whether they were at high or low risk., Results: Of 1286 trials from 205 meta-analyses, 556 (43%) had at least one domain at high risk of bias. Among the sample of 200 of these trials, 142 were confirmed as high risk; in these, we identified 25 types of methodological problem. Adjustments were possible in 136 trials (96%). Easy adjustments with no or minor cost could be applied in 71 trials (50%), resulting in 17 trials (12%) changing to low risk for all domains. So the avoidable waste represented 12% (95% CI 7% to 18%) of trials with at least one domain at high risk. After correcting for incomplete reporting, avoidable waste due to inadequate methods was estimated at 42% (95% CI 36% to 49%)., Conclusions: An important burden of wasted research is related to inadequate methods. This waste could be partly avoided by simple and inexpensive adjustments., (© Yordanov et al 2015.)

Background: To investigate the nature of the research process as a whole, factors that might influence the way in which research is carried out, and how researchers ultimately report their findings., Methods: Semi-structured qualitative telephone interviews with authors of trials, identified from two sources: trials published since 2002 included in Cochrane systematic reviews selected for the ORBIT project; and trial reports randomly sampled from 14,758 indexed on PubMed over the 12-month period from August 2007 to July 2008., Results: A total of 268 trials were identified for inclusion, 183 published since 2002 and included in the Cochrane systematic reviews selected for the ORBIT project and 85 randomly selected published trials indexed on PubMed. The response rate from researchers in the former group was 21% (38/183) and in the latter group was 25% (21/85). Overall, 59 trialists were interviewed from the two different sources. A number of major but related themes emerged regarding the conduct and reporting of trials: establishment of the research question; identification of outcome variables; use of and adherence to the study protocol; conduct of the research; reporting and publishing of findings. Our results reveal that, although a substantial proportion of trialists identify outcome variables based on their clinical experience and knowing experts in the field, there can be insufficient reference to previous research in the planning of a new trial. We have revealed problems with trial recruitment: not reaching the target sample size, over-estimation of recruitment potential and recruiting clinicians not being in equipoise. We found a wide variation in the completeness of protocols, in terms of detailing study rationale, outlining the proposed methods, trial organisation and ethical considerations., Conclusion: Our results confirm that the conduct and reporting of some trials can be inadequate. Interviews with researchers identified aspects of clinical research that can be especially challenging: establishing appropriate and relevant outcome variables to measure, use of and adherence to the study protocol, recruiting of study participants and reporting and publishing the study findings. Our trialists considered the prestige and impact factors of academic journals to be the most important criteria for selecting those to which they would submit manuscripts.

Background: Calls have been made for increased access to individual participant data (IPD) from clinical trials, to ensure that complete evidence is available. However, despite the obvious benefits, progress towards this is frustratingly slow. In the meantime, many systematic reviews have already collected IPD from clinical trials. We propose that a central repository for these IPD should be established to ensure that these datasets are safeguarded and made available for use by others, building on the strengths and advantages of the collaborative groups that have been brought together in developing the datasets., Objective: Evaluate the level of support, and identify major issues, for establishing a central repository of IPD., Design: On-line survey with email reminders., Participants: 71 reviewers affiliated with the Cochrane Collaboration's IPD Meta-analysis Methods Group were invited to participate., Results: 30 (42%) invitees responded: 28 (93%) had been involved in an IPD review and 24 (80%) had been involved in a randomised trial. 25 (83%) agreed that a central repository was a good idea and 25 (83%) agreed that they would provide their IPD for central storage. Several benefits of a central repository were noted: safeguarding and standardisation of data, increased efficiency of IPD meta-analyses, knowledge advancement, and facilitating future clinical, and methodological research. The main concerns were gaining permission from trial data owners, uncertainty about the purpose of the repository, potential resource implications, and increased workload for IPD reviewers. Restricted access requiring approval, data security, anonymisation of data, and oversight committees were highlighted as issues under governance of the repository., Conclusion: There is support in this community of IPD reviewers, many of whom are also involved in clinical trials, for storing IPD in a central repository. Results from this survey are informing further work on developing a repository of IPD which is currently underway by our group.

Without a complete published description of interventions, clinicians and patients cannot reliably implement interventions that are shown to be useful, and other researchers cannot replicate or build on research findings. The quality of description of interventions in publications, however, is remarkably poor. To improve the completeness of reporting, and ultimately the replicability, of interventions, an international group of experts and stakeholders developed the Template for Intervention Description and Replication (TIDieR) checklist and guide. The process involved a literature review for relevant checklists and research, a Delphi survey of an international panel of experts to guide item selection, and a face to face panel meeting. The resultant 12 item TIDieR checklist (brief name, why, what (materials), what (procedure), who provided, how, where, when and how much, tailoring, modifications, how well (planned), how well (actual)) is an extension of the CONSORT 2010 statement (item 5) and the SPIRIT 2013 statement (item 11). While the emphasis of the checklist is on trials, the guidance is intended to apply across all evaluative study designs. This paper presents the TIDieR checklist and guide, with an explanation and elaboration for each item, and examples of good reporting. The TIDieR checklist and guide should improve the reporting of interventions and make it easier for authors to structure accounts of their interventions, reviewers and editors to assess the descriptions, and readers to use the information.

Over the past three decades it has become increasingly recognized that systematic assessment of as high a proportion as possible of relevant research evidence is needed to protect the best interests of patients and the public. For example, this principle is manifested in clinical guidelines and, increasingly, in the design and monitoring of new research. For scientific and ethical reasons, those responsible for monitoring the progress of ongoing clinical trials may need to seek unpublished and interim data to protect the interests of actual or potential participants in research. The challenge facing data monitoring committees has received relatively little attention, however. In this paper we review some of the commentaries on the issue and the few accounts of actual data monitoring committee experiences. We then present details of our own recent experience as members of the data monitoring committee for the BOOST-II UK trial (ISRCTN:0084226), one of five concurrent trials assessing the level of arterial oxygen which should be targeted in the care of very premature neonates. We conclude that efficient protection both of the interests of actual or potential participants in research and of science requires that data monitoring committees have access to all relevant research, including unpublished and interim data.

The protocol of a clinical trial serves as the foundation for study planning, conduct, reporting, and appraisal. However, trial protocols and existing protocol guidelines vary greatly in content and quality. This article describes the systematic development and scope of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013, a guideline for the minimum content of a clinical trial protocol.The 33-item SPIRIT checklist applies to protocols for all clinical trials and focuses on content rather than format. The checklist recommends a full description of what is planned; it does not prescribe how to design or conduct a trial. By providing guidance for key content, the SPIRIT recommendations aim to facilitate the drafting of high-quality protocols. Adherence to SPIRIT would also enhance the transparency and completeness of trial protocols for the benefit of investigators, trial participants, patients, sponsors, funders, research ethics committees or institutional review boards, peer reviewers, journals, trial registries, policymakers, regulators, and other key stakeholders., Competing Interests: Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M12-1905.

High quality protocols facilitate proper conduct, reporting, and external review of clinical trials. However, the completeness of trial protocols is often inadequate. To help improve the content and quality of protocols, an international group of stakeholders developed the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials). The SPIRIT Statement provides guidance in the form of a checklist of recommended items to include in a clinical trial protocol. This SPIRIT 2013 Explanation and Elaboration paper provides important information to promote full understanding of the checklist recommendations. For each checklist item, we provide a rationale and detailed description; a model example from an actual protocol; and relevant references supporting its importance. We strongly recommend that this explanatory paper be used in conjunction with the SPIRIT Statement. A website of resources is also available (www.spirit-statement.org). The SPIRIT 2013 Explanation and Elaboration paper, together with the Statement, should help with the drafting of trial protocols. Complete documentation of key trial elements can facilitate transparency and protocol review for the benefit of all stakeholders.

The selection of appropriate outcomes or domains is crucial when designing clinical trials in order to compare directly the effects of different interventions in ways that minimize bias. If the findings are to influence policy and practice then the chosen outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. There is a growing recognition that insufficient attention has been paid to the outcomes measured in clinical trials. These issues could be addressed through the development and use of an agreed standardized collection of outcomes, known as a core outcome set, which should be measured and reported, as a minimum, in all trials for a specific clinical area. Accumulating work in this area has identified the need for general guidance on the development of core outcome sets. Key issues to consider in the development of a core outcome set include its scope, the stakeholder groups to involve, choice of consensus method and the achievement of a consensus.

Two recent publications from Helsinki and Toronto that investigated the natural history of brain AVMs are the background topic for reviewing some principles and pitfalls of prognostic studies. Multivariable prognostic research involves 3 steps: developing the prognostic model, validating its performance in other individuals, and assessing its clinical impact on patients' outcomes. Unfortunately, the predictive ability of the model can be poor when it is applied to a new population, and clinical impact studies are rarely performed. Models that have not been validated should not be used to inform clinical decisions. Unfortunately, for rare outcomes in rare diseases, clinical data are limited. Although the 2 studies on brain AVMs may represent the best data currently available, they still included few patients with events and there are several methodologic concerns undermining the reliability of results. The estimates of risk of rupture per year are uncertain. Multiplying those uncertain numbers by the life expectancy of individuals can inflate error beyond control. Hence relying on these estimates to make clinical decisions may be dangerous.

Objectives: To provide information on the frequency and reasons for outcome reporting bias in clinical trials., Design: Trial protocols were compared with subsequent publication(s) to identify any discrepancies in the outcomes reported, and telephone interviews were conducted with the respective trialists to investigate more extensively the reporting of the research and the issue of unreported outcomes., Participants: Chief investigators, or lead or coauthors of trials, were identified from two sources: trials published since 2002 covered in Cochrane systematic reviews where at least one trial analysed was suspected of being at risk of outcome reporting bias (issue 4, 2006; issue 1, 2007, and issue 2, 2007 of the Cochrane library); and a random sample of trial reports indexed on PubMed between August 2007 and July 2008., Setting: Australia, Canada, Germany, the Netherlands, New Zealand, the United Kingdom, and the United States., Main Outcome Measures: Frequency of incomplete outcome reporting-signified by outcomes that were specified in a trial's protocol but not fully reported in subsequent publications-and trialists' reasons for incomplete reporting of outcomes., Results: 268 trials were identified for inclusion (183 from the cohort of Cochrane systematic reviews and 85 from PubMed). Initially, 161 respective investigators responded to our requests for interview, 130 (81%) of whom agreed to be interviewed. However, failure to achieve subsequent contact, obtain a copy of the study protocol, or both meant that final interviews were conducted with 59 (37%) of the 161 trialists. Sixteen trial investigators failed to report analysed outcomes at the time of the primary publication, 17 trialists collected outcome data that were subsequently not analysed, and five trialists did not measure a prespecified outcome over the course of the trial. In almost all trials in which prespecified outcomes had been analysed but not reported (15/16, 94%), this under-reporting resulted in bias. In nearly a quarter of trials in which prespecified outcomes had been measured but not analysed (4/17, 24%), the "direction" of the main findings influenced the investigators' decision not to analyse the remaining data collected. In 14 (67%) of the 21 randomly selected PubMed trials, there was at least one unreported efficacy or harm outcome. More than a quarter (6/21, 29%) of these trials were found to have displayed outcome reporting bias., Conclusion: The prevalence of incomplete outcome reporting is high. Trialists seemed generally unaware of the implications for the evidence base of not reporting all outcomes and protocol changes. A general lack of consensus regarding the choice of outcomes in particular clinical settings was evident and affects trial design, conduct, analysis, and reporting.

The Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) were published in five journals in 2001 and 2002. These guidelines, in the form of a checklist and explanations for use by authors and journal editors, were designed to improve reporting of acupuncture trials, particularly the interventions, thereby facilitating their interpretation and replication. Subsequent reviews of the application and impact of STRICTA have highlighted the value of STRICTA as well as scope for improvements and revision. To manage the revision process a collaboration between the STRICTA Group, the CONSORT Group, and the Chinese Cochrane Centre was developed in 2008. An expert panel with 47 participants was convened that provided electronic feedback on a revised draft of the checklist. At a subsequent face-to-face meeting in Freiburg, a group of 21 participants further revised the STRICTA checklist and planned dissemination. The new STRICTA checklist, which is an official extension of CONSORT, includes six items and 17 sub-items. These set out reporting guidelines for the acupuncture rationale, the details of needling, the treatment regimen, other components of treatment, the practitioner background, and the control or comparator interventions. In addition, and as part of this revision process, the explanations for each item have been elaborated, and examples of good reporting for each item are provided. In addition, the word "controlled" in STRICTA is replaced by "clinical," to indicate that STRICTA is applicable to a broad range of clinical evaluation designs, including uncontrolled outcome studies and case reports. It is intended that the revised STRICTA, in conjunction with both the main CONSORT Statement and extension for nonpharmacologic treatment, will raise the quality of reporting of clinical trials of acupuncture.

The STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) were published in five journals in 2001 and 2002. These guidelines, in the form of a checklist and explanations for use by authors and journal editors, were designed to improve reporting of acupuncture trials, particularly the interventions, thereby facilitating their interpretation and replication. Subsequent reviews of the application and impact of STRICTA have highlighted the value of STRICTA as well as scope for improvements and revision. To manage the revision process a collaboration between the STRICTA Group, the CONSORT Group and the Chinese Cochrane Centre was developed in 2008. An expert panel with 47 participants was convened that provided electronic feedback on a revised draft of the checklist. At a subsequent face-to-face meeting in Freiburg, a group of 21 participants further revised the STRICTA checklist and planned dissemination. The new STRICTA checklist, which is an official extension of CONSORT, includes 6 items and 17 subitems. These set out reporting guidelines for the acupuncture rationale, the details of needling, the treatment regimen, other components of treatment, the practitioner background and the control or comparator interventions. In addition, and as part of this revision process, the explanations for each item have been elaborated, and examples of good reporting for each item are provided. In addition, the word "controlled" in STRICTA is replaced by "clinical", to indicate that STRICTA is applicable to a broad range of clinical evaluation designs, including uncontrolled outcome studies and case reports. It is intended that the revised STRICTA checklist, in conjunction with both the main CONSORT statement and extension for non-pharmacological treatment, will raise the quality of reporting of clinical trials of acupuncture.

Mounting evidence suggests that there is frequently considerable variation in the risk of the outcome of interest in clinical trial populations. These differences in risk will often cause clinically important heterogeneity in treatment effects (HTE) across the trial population, such that the balance between treatment risks and benefits may differ substantially between large identifiable patient subgroups; the "average" benefit observed in the summary result may even be non-representative of the treatment effect for a typical patient in the trial. Conventional subgroup analyses, which examine whether specific patient characteristics modify the effects of treatment, are usually unable to detect even large variations in treatment benefit (and harm) across risk groups because they do not account for the fact that patients have multiple characteristics simultaneously that affect the likelihood of treatment benefit. Based upon recent evidence on optimal statistical approaches to assessing HTE, we propose a framework that prioritizes the analysis and reporting of multivariate risk-based HTE and suggests that other subgroup analyses should be explicitly labeled either as primary subgroup analyses (well-motivated by prior evidence and intended to produce clinically actionable results) or secondary (exploratory) subgroup analyses (performed to inform future research). A standardized and transparent approach to HTE assessment and reporting could substantially improve clinical trial utility and interpretability.

In 22 of 44 industry-initiated clinical trial protocols from 1994-95, it was noted that the sponsor either owned the data or needed to approve the manuscript; another 18 protocols had other constraints. Furthermore, in 16 trials, the sponsor had access to accumulating data, and in an additional 16 trials the sponsor could stop the trial at any time, for any reason. These facts were not noted in any of the trial reports. We found similar constraints on publication rights in 44 protocols from 2004. This tight sponsor control over industry-initiated trials should be changed.

Objectives: To consider methods and related evidence for evaluating bias in non-randomised intervention studies., Data Sources: Systematic reviews and methodological papers were identified from a search of electronic databases; handsearches of key medical journals and contact with experts working in the field. New empirical studies were conducted using data from two large randomised clinical trials., Methods: Three systematic reviews and new empirical investigations were conducted. The reviews considered, in regard to non-randomised studies, (1) the existing evidence of bias, (2) the content of quality assessment tools, (3) the ways that study quality has been assessed and addressed. (4) The empirical investigations were conducted generating non-randomised studies from two large, multicentre randomised controlled trials (RCTs) and selectively resampling trial participants according to allocated treatment, centre and period., Results: In the systematic reviews, eight studies compared results of randomised and non-randomised studies across multiple interventions using meta-epidemiological techniques. A total of 194 tools were identified that could be or had been used to assess non-randomised studies. Sixty tools covered at least five of six pre-specified internal validity domains. Fourteen tools covered three of four core items of particular importance for non-randomised studies. Six tools were thought suitable for use in systematic reviews. Of 511 systematic reviews that included non-randomised studies, only 169 (33%) assessed study quality. Sixty-nine reviews investigated the impact of quality on study results in a quantitative manner. The new empirical studies estimated the bias associated with non-random allocation and found that the bias could lead to consistent over- or underestimations of treatment effects, also the bias increased variation in results for both historical and concurrent controls, owing to haphazard differences in case-mix between groups. The biases were large enough to lead studies falsely to conclude significant findings of benefit or harm. Four strategies for case-mix adjustment were evaluated: none adequately adjusted for bias in historically and concurrently controlled studies. Logistic regression on average increased bias. Propensity score methods performed better, but were not satisfactory in most situations. Detailed investigation revealed that adequate adjustment can only be achieved in the unrealistic situation when selection depends on a single factor., Conclusions: Results of non-randomised studies sometimes, but not always, differ from results of randomised studies of the same intervention. Non-randomised studies may still give seriously misleading results when treated and control groups appear similar in key prognostic factors. Standard methods of case-mix adjustment do not guarantee removal of bias. Residual confounding may be high even when good prognostic data are available, and in some situations adjusted results may appear more biased than unadjusted results. Although many quality assessment tools exist and have been used for appraising non-randomised studies, most omit key quality domains. Healthcare policies based upon non-randomised studies or systematic reviews of non-randomised studies may need re-evaluation if the uncertainty in the true evidence base was not fully appreciated when policies were made. The inability of case-mix adjustment methods to compensate for selection bias and our inability to identify non-randomised studies that are free of selection bias indicate that non-randomised studies should only be undertaken when RCTs are infeasible or unethical. Recommendations for further research include: applying the resampling methodology in other clinical areas to ascertain whether the biases described are typical; developing or refining existing quality assessment tools for non-randomised studies; investigating how quality assessments of non-randomised studies can be incorporated into reviews and the implications of individual quality features for interpretation of a review's results; examination of the reasons for the apparent failure of case-mix adjustment methods; and further evaluation of the role of the propensity score.

Among clinical trials assessing a given treatment, often parallel and cross-over designs are used together. In the first paper of a series of three, we explore two methods to pool continuous outcomes in a meta-analysis combining parallel and cross-over trial designs: the weighted mean difference (WMD) and the standardized weighted mean difference (SWMD). The combined design meta-analytic formulae are based on a weighted average of the two design treatment estimates. A random effects model can be implemented. Both WMD and SWMD can be used, the choice of the method is determined by the type of outcomes obtained in the trials. Compared to the number of included subjects, the relative weight of the cross-over design is large in combined-design meta-analysis. Differences in the weight estimation between WMD and SWMD can also accentuate the relative weight of cross-over trials, which must be considered a case of design-specific bias., (Copyright 2002 John Wiley & Sons, Ltd.)

We examine different methods to pool binary outcomes used both in parallel and cross-over trials. Odds ratio (OR) estimators obtained from joint conditional probabilities in cross-over trials, such as the Mantel-Haenszel and Peto methods, are compared to an OR estimator using marginal results of cross-over trials. When there is correlation between the outcomes in the two cross-over periods, joint conditional ORs differ from marginal ORs and cannot be combined with OR estimates from parallel trials. The marginal OR estimate is independent of the between-period correlation and it includes a correction for cross-over correlation in the variance estimate. As its computation is similar in cross-over and parallel trials, it is the method of choice to pool results from parallel and cross-over trials in a combined design meta-analysis., (Copyright 2002 John Wiley & Sons, Ltd.)