Search

Background: Adverse events (AEs) in randomized clinical trials may be reported in multiple sources. Different methods for reporting adverse events across trials, or across sources for a single trial, may produce inconsistent and confusing information about the adverse events associated with interventions Methods: We sought to compare the methods authors use to decide which AEs to include in a particular source (i.e., “selection criteria”) and to determine how selection criteria could impact the AEs reported. We compared sources (e.g., journal articles, clinical study reports [CSRs]) of trials for two drug-indications: gabapentin for neuropathic pain and quetiapine for bipolar depression. We identified selection criteria and assessed how criteria affected AE reporting. Results: We identified 21 gabapentin trials and 7 quetiapine trials. All CSRs (6 gabapentin, 2 quetiapine) reported all AEs without applying selection criteria; by comparison, no other source reported all AEs, and 15/68 (22%) gabapentin sources and 19/48 (40%) quetiapine sources reported using selection criteria. Selection criteria greatly affected the number of AEs that would be reported. For example, 67/316 (21%) AEs in one quetiapine trial met the criterion “occurring in ≥2% of participants in any treatment group,” while only 5/316 (2%) AEs met the criterion, “occurring in ≥10% of quetiapine-treated patients and twice as frequent in the quetiapine group as the placebo group.” Conclusions: Selection criteria for reporting AEs vary across trials and across sources for individual trials. If investigators do not pre-specify selection criteria, they might “cherry-pick” AEs based on study results. Even if investigators pre-specify selection criteria, selective reporting of AEs will produce biased meta-analyses and clinical practice guidelines. Data about all AEs identified in clinical trials should be publicly available; however, sharing data will not solve all the problems we identified in this study. Keywords: Harms, adverse events, clinical trials, reporting bias, selective outcome reporting, data sharing, trial registration

Clinical trials and systematic reviews of clinical trials inform healthcare decisions. There is growing concern, however, about results from clinical trials that cannot be reproduced. Reasons for nonreproducibility include that outcomes are defined in multiple ways, results can be obtained using multiple methods of analysis, and trial findings are reported in multiple sources (“multiplicity”). Multiplicity combined with selective reporting can influence dissemination of trial findings and decision-making. In particular, users of evidence might be misled by exposure to selected sources and overly optimistic representations of intervention effects. In this commentary, drawing from our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study and evidence from previous research, we offer practical recommendations to enhance the reproducibility of clinical trials and systematic reviews.