Search

PurposeTrustworthy reporting of quadrivalent human papillomavirus (HPV) vaccine trials is the foundation for assessing the vaccine’s risks and benefits. However, several pivotal trial publications incompletely reported important methodological details and inaccurately described the formulation that the control arms received. Under the Restoring Invisible and Abandoned Trials initiative (RIAT), we aim to restore the public record regarding the content and rationale of the controls used in the trials.MethodsWe assembled a cohort (five randomised controlled trials) described as placebo-controlled using clinical study reports (CSRs) obtained from the European Medicines Agency. We extracted the content and rationale for the choice of control used in each trial across six data sources: trial publications, register records, CSR synopses, CSR main bodies, protocols and informed consent forms.ResultsAcross data sources, the control was inconsistently reported as ‘placebo’-containing aluminium adjuvant (sometimes with dose information). Amorphous aluminium hydroxyphosphate sulfate (AAHS) was not mentioned in any trial registry entry, but was mentioned in all publications and CSRs. In three of five trials, consent forms described the control as an ‘inactive’ substance. No rationale for the selection of the control was reported in any trial publication, register, consent form, CSR synopsis or protocol. Three trials reported the rationale for choice of control in CSRs: to preserve blinding and assess the safety of HPV virus-like particles as the ‘safety profile of (AAHS) is well characterised’.ConclusionsThe stated rationale of using AAHS control—to characterise the safety of the HPV virus-like particles—lacks clinical relevance. A non-placebo control may have obscured an accurate assessment of safety and the participant consent process of some trials raises ethical concerns.Trial registration numbersNCT00092482,NCT00092521,NCT00092534,NCT00090220,NCT00090285.

Background: To improve reporting transparency and research integrity, some journals have begun publishing study protocols and statistical analysis plans alongside trial publications. We sought to assess the overall availability and characteristics of protocols and statistical analysis plans of randomized clinical trials published in the top five (by impact factor) general medicine journals. Methods: All randomized clinical trials published in Annals of Internal Medicine, BMJ, JAMA, Lancet, and NEJM in 2016 were identified. For each randomized clinical trial, we searched for protocols and statistical analysis plans on journal websites (including supplementary material) and in the article, for example, a referenced publication or link to trial or institutional website. Characteristics of randomized clinical trials were extracted from the publication and clinical trial registry. A detailed assessment of protocols and statistical analysis plans was conducted in a 20% random sample of randomized clinical trials. Results: Protocols were available for 299 (82%) trials, ranging from 50% in BMJ to >95% in NEJM and JAMA. Statistical analysis plans were available for 182 (50%) trials and varied from Conclusion: Protocols and statistical analysis plans are publicly available for the majority of trials. However, the a priori versions of these documents are only available for a minority of trials. More attention must be paid to ensuring the public availability of a priori versions.

After rollout under emergency authorisation, manufacturers of covid-19 vaccines now have their sights on regulatory approval. But what’s the rush, asks Peter Doshi, and is just six months of data from now unblinded trials acceptable?

Objective: We examined nonsystematic adverse events (AEs) in Part 2 (of 2) of a study describing the assessment and reporting AEs in clinical trials. Study Design and Setting: We examined 21 trials of gabapentin for neuropathic pain (52 sources) and seven trials of quetiapine for bipolar depression (80 sources) using data from the Multiple Data Sources study. We extracted and compared information about nonsystematic AEs (i.e., AEs that were not assessed for every participant), including AEs categorized as “serious.” We recorded whether AEs were grouped by anatomic or physiological system. Results: Trials of the same drug reported information about different AEs. Information in public sources was inadequate for decision-making. No public source reported all AEs, or all serious AEs, identified in nonpublic sources about the same trial. Of trials with only public sources, 2/15 (13%) gabapentin and 0/3 (0%) quetiapine trials grouped AEs by anatomic or physiological system. Conclusion: Public sources contained little information about nonsystematic AEs, including serious AEs. Grouping might make nonsystematic AEs easier to detect; however, most public sources did not report grouped AEs. Standards are needed to improve the collection and reporting of nonsystematic AEs so that stakeholders can use trials to assess the balance of potential benefits and harms.

Background Participants are recruited into clinical trials under the assumption that the research will contribute to medical knowledge. Therefore, non-publication trials—and, more recently, lack of data sharing—are widely considered to violate the trust of trial participants. Existing practices regarding patient consent to publication and data sharing have not been evaluated. Analyzing informed consent forms (ICFs), we studied what trial participants were told regarding investigators’ intention to contribute to medical knowledge, publish trial results, and share de-identified trial data. Methods We obtained 98 ICFs of industry-funded pre-marketing trials for all (17) antibiotics approved by the European Medicines Agency and 46 ICFs of publicly funded trials from the National Heart, Lung and Blood Institute Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC) data repository. Three authors independently reviewed ICFs to identify and extract what was stated or implied regarding: (1) publication of results; (2) sharing de-identified data; (3) data ownership; (4) confidentiality of identifiable data; and (5) whether the trial will produce knowledge that offers public benefit. Consensus was obtained from the two reviewers with the greatest overall agreement on all five measures. Disagreements were resolved through discussion among all authors. Results Four (3%) trials indicated a commitment to publish trial results; 140 (97%) did not commit to publishing trial results; six (4%) indicated a commitment to share de-identified data with third party researchers. Commitments to share were more common in publicly funded trials than industry-funded trials (7% vs 3%). A total of 103 (72%) ICFs indicated the trials will or may produce knowledge that offers public benefits, while 131 (91%) ICFs left unstated who “owned” trial data; of those with statements, the sponsor always claimed ownership. Patient confidentiality was guaranteed in 137 (95%) trials. Conclusions Our results suggest that consent forms rarely disclose investigators’ intentions regarding the sharing of de-identified data or publication of trial results.

Objectives The objective of this study was to determine whether disagreements among multiple data sources affect systematic reviews of randomized clinical trials (RCTs). Study Design and Setting Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic sources (clinical study reports [CSRs] and individual participant data [IPD]). Results We found 21 gabapentin RCTs (74 reports, 6 IPD) and 7 quetiapine RCTs (50 reports, 1 IPD); most were reported in journal articles (18/21 [86%] and 6/7 [86%], respectively). When available, CSRs contained the most trial design and risk of bias information. CSRs and IPD contained the most results. For the outcome domains “pain intensity” (gabapentin) and “depression” (quetiapine), we found single trials with 68 and 98 different meta-analyzable results, respectively; by purposefully selecting one meta-analyzable result for each RCT, we could change the overall result for pain intensity from effective (standardized mean difference [SMD] = −0.45; 95% confidence interval [CI]: −0.63 to −0.27) to ineffective (SMD = −0.06; 95% CI: −0.24 to 0.12). We could change the effect for depression from a medium effect (SMD = −0.55; 95% CI: −0.85 to −0.25) to a small effect (SMD = −0.26; 95% CI: −0.41 to −0.1). Conclusions Disagreements across data sources affect the effect size, statistical significance, and interpretation of trials and meta-analyses.

Purpose Regulators approve written medical information for healthcare professionals and consumers, but the consistency of these sources has not been studied. We investigated the consistency of information regarding four cardiovascular risks of attention-deficit/hyperactivity disorder (ADHD) medications approved in four countries. Methods Professional and consumer product labeling for five ADHD medications approved in Australia, Canada, the UK, and the USA were obtained in March/April 2016. Language describing the relationship between medication and elevated blood pressure and/or heart rate, myocardial infarction, stroke, and sudden death was extracted verbatim and classified into one of four categories based on the described relationship between medication and adverse event: “confirmed,” “unconfirmed,” “mixed,” and “not mentioned.” We judged the consistency of messages delivered to healthcare professionals and consumers as either “consistent” or “inconsistent.” Results We obtained 20 healthcare professional labels and 20 corresponding consumer labels for the five ADHD medications registered in all four countries. Not all professional and consumer labeling contained language regarding all four adverse events. Of the 80 theoretically evaluable drug-risk pairs, 38 (48%) were not evaluable because of absence of mention of the adverse event in the consumer label. For the remaining 42, the potential causal relationship was expressed consistently in professional and consumer labeling in 25 (60%) cases. The cardiovascular risk profile was not described consistently across all four countries for any of the five drugs. Conclusions Product labeling provides healthcare professionals and consumers with inconsistent messages regarding the potential causal relationship between stimulant use and specific cardiovascular risks in children and adolescents. Copyright © 2017 John Wiley & Sons, Ltd.

Ethical, evidence-informed decision making is undermined by the grave concerns that have emerged over the trustworthiness of clinical trials published in biomedical journals. The inescapable conclusion from this growing body of research is that what we see, even in the most highly regarded peer-reviewed journals, cannot be trusted at face value. Concerns of inaccurate, biased, and insufficient reporting of trials are impossible to resolve without access to underlying trial data. Access to such data, including things like clinical study reports—huge, unabridged, detailed reports of clinical trials—would minimise the risk of distortions and selective publication. But the FDA, the world’s greatest custodian of those data, just sits on them. We see no reason why FDA should not publicly release clinical study reports with minimal redactions. The European regulator is already doing this, but FDA’s holdings are far greater. Data transparency is not simply an “opportunity” FDA might consider, but rather an ethical imperative. The Blueprint is good but does not go far enough. We do not need gates, barriers and committees between us and access to aggregate reports on drugs and other interventions which we are prescribing or using daily. Let’s leave the nannies at home.

The world has bet the farm on vaccines as the solution to the pandemic, but the trials are not focused on answering the questions many might assume they are. Peter Doshi reports

It seemed a truth universally acknowledged that the human population had no pre-existing immunity to SARS-CoV-2, but is that actually the case? Peter Doshi explores the emerging research on immunological responses

Experts are troubled by the apparent lack of informed consent in a large, cluster randomised study of the malaria vaccine. Peter Doshi reports

After three years of trying to access trial data for HPV vaccines, Lars Jorgensen and colleagues find current transparency policies unfit for their purpose

Reporting bias is a major threat to the validity and credibility of systematic reviews. This article outlines the rationale for accessing clinical study reports and other regulatory documents (regulatory data) as a means of addressing reporting bias and identifies factors that may help decide whether (or not) to include regulatory data in systematic reviews. The article also describes the origins and current state of regulatory data access and summarises a survey of current systematic reviewers’ practices in considering regulatory data for inclusion in systematic reviews. How to access and extract regulatory data is not addressed. Organisations and other stakeholders such as Cochrane should encourage the use of data from clinical study reports as an important source of data in reviews of pharmaceutical interventions particularly when the intervention in question is of high importance and the risk of reporting bias is great.

Background Clinical study reports (CSRs) are produced for marketing authorisation applications. They often contain considerably more information about, and data from, clinical trials than corresponding journal publications. Use of data from CSRs might help circumvent reporting bias, but many researchers appear to be unaware of their existence or potential value. Our survey aimed to gain insight into the level of familiarity, understanding and use of CSRs, and to raise awareness of their potential within the systematic review community. We also aimed to explore the potential barriers faced when obtaining and using CSRs in systematic reviews. Methods Online survey of systematic reviewers who (i) had requested or used CSRs, (ii) had considered but not used CSRs and (iii) had not considered using CSRs was conducted. Cochrane reviewers were contacted twice via the Cochrane monthly digest. Non-Cochrane reviewers were reached via journal and other website postings. Results One hundred sixty respondents answered an open invitation and completed the questionnaire; 20/160 (13%) had previously requested or used CSRs and other regulatory documents, 7/160 (4%) had considered but not used CSRs and 133/160 (83%) had never considered this data source. Survey respondents mainly sought data from the European Medicines Agency (EMA) and/or the Food and Drug Administration (FDA). Motivation for using CSRs stemmed mainly from concerns about reporting bias 11/20 (55%), specifically outcome reporting bias 11/20 (55%) and publication bias 5/20 (25%). The barriers to using CSRs noted by all types of respondents included current limited access to these documents (43 respondents), the time and resources needed to obtain and include these data in evidence syntheses (n = 25) and lack of guidance about how to use these sources in systematic reviews (n = 26). Conclusions Most respondents (irrespective of whether they had previously used them) agreed that access to CSRs is important, and suggest that further guidance on how to use and include these data would help to promote their use in future systematic reviews. Most respondents who received CSRs considered them to be valuable in their systematic review and/or meta-analysis. Electronic supplementary material The online version of this article (10.1186/s13643-018-0766-x) contains supplementary material, which is available to authorized users.

As the EMA’s recommendation to suspend HES solutions from the market heads to the European Commission for a final decision, scientists who were asked to provide independent advice say that they were completely ignored. Peter Doshi reports

BackgroundNeuraminidase inhibitors (NIs) are stockpiled and recommended by public health agencies for treating and preventing seasonal and pandemic influenza. They are used clinically worldwide.ObjectivesTo (1) describe the potential benefits and harms of NIs for influenza in all age groups by reviewing all clinical study reports (CSRs) of published and unpublished randomised, placebo-controlled trials and regulatory comments; and (2) determine the effect of oseltamivir (Tamiflu®, Roche) treatment on mortality in patients with 2009A/H1N1 influenza.MethodsWe searched trial registries, electronic databases and corresponded with regulators and sponsors to identify randomised trials of NIs. We requested full CSRs and accessed regulators’ comments. We included only those trials for which we had CSRs. To examine the effects of oseltamivir on 2009A/H1N1 influenza mortality, we requested individual patient data (IPD) from corresponding authors of all included observational studies.ResultsEffect of oseltamivir and zanamivir (Relenza®, GlaxoSmithKline) in the prevention and treatment of influenza: Oseltamivir reduced the time to first alleviation of symptoms in adults by 16.8 hours [95% confidence interval (CI) 8.4 to 25.1 hours]. Zanamivir reduced the time to first alleviation of symptoms in adults by 0.60 days (95% CI 0.39 to 0.81 days). Oseltamivir reduced unverified pneumonia in adult treatment [risk difference (RD) 1.00%, 95% CI 0.22% to 1.49%]; similar findings were observed with zanamivir prophylaxis in adults (RD 0.32%, 95% CI 0.09% to 0.41%). Oseltamivir treatment of adults increased the risk of nausea (RD 3.66%, 95% CI 0.90% to 7.39%) and vomiting (RD 4.56%, 95% CI 2.39% to 7.58%). In the treatment of children, oseltamivir induced vomiting (RD 5.34%, 95% CI 1.75% to 10.29%). Both oseltamivir and zanamivir prophylaxis reduced the risk of symptomatic influenza in individuals (oseltamivir RD 3.05%, 95% CI 1.83% to 3.88%; zanamivir RD 1.98%, 95% CI 0.98% to 2.54%) and in households (oseltamivir RD 13.6%, 95% CI 9.52% to 15.47%; zanamivir RD 14.84%, 95% CI 12.18% to 16.55%). Oseltamivir increased psychiatric adverse events in the combined on- and off-treatment periods (RD 1.06%, 95% CI 0.07% to 2.76%) and the risk of headaches while on treatment (RD 3.15%, 95% CI 0.88% to 5.78%).Effect of oseltamivir on mortality in patients with 2009A/H1N1 influenza: Analysis of summary data of 30 studies as well as IPD of four studies showed evidence of time-dependent bias. After adjusting for time-dependent bias and potential confounding variables, competing risks analysis of the IPD showed insufficient evidence that oseltamivir reduced the risk of mortality (hazard ratio 1.03, 95% CI 0.64 to 1.65).ConclusionsOseltamivir and zanamivir cause small reductions in the time to first alleviation of influenza symptoms in adults. The use of oseltamivir increases the risk of nausea, vomiting, psychiatric events in adults and vomiting in children. Oseltamivir has no protective effect on mortality among patients with 2009A/H1N1 influenza. Prophylaxis with either NI may reduce symptomatic influenza in individuals and in households. The balance between benefits and harms should be considered when making decisions about use of NIs for either prophylaxis or treatment of influenza.Study registrationThis study is registered as PROSPERO CRD42012002245.FundingThe National Institute for Health Research Health Technology Assessment programme.

Vaccines are considered one of public health’s greatest success stories. But is all promotion of vaccines necessarily a good thing, or does it depend on the details? Peter Doshi investigates the semi-transparent world of vaccine advocacy organizations

Importance Potential research participants may assume that randomized trials comparing new interventions with older interventions always hypothesize greater efficacy for the new intervention, as in superiority trials. However, antibiotic trials frequently use “noninferiority” hypotheses allowing a degree of inferior efficacy deemed “clinically acceptable” compared with an older effective drug, in exchange for nonefficacy benefits (eg, decreased adverse effects). Considering these different benefit-harm trade-offs, proper informed consent necessitates supplying different information on the purposes of superiority and noninferiority trials. Objective To determine the degree to which the study purpose is explained to potential participants in randomized clinical trials of antibiotics and the degree to which study protocols justify their selection of noninferiority hypotheses and amount of “clinically acceptable” inferiority. Design and Setting Cross-sectional analysis of study protocols, statistical analysis plans (SAPs), and informed consent forms (ICFs) from clinical study reports submitted to the European Medicines Agency. The ICFs were read by both methodologists and patient investigators. Main Outcomes and Measures Protocols and SAPs were used as the reference standard to determine prespecified primary hypothesis and record rationale for selection of noninferiority hypotheses and noninferiority margins. This information was cross-referenced against ICFs to determine whether ICFs explained the study purpose. Results We obtained trial documents from 78 randomized trials with prespecified efficacy hypotheses (6 superiority, 72 noninferiority) for 17 antibiotics conducted between 1991 and 2011 that enrolled 39 407 patients. Fifty were included in the ICF analysis. All ICFs contained sections describing study purpose; however, none consistently conveyed study hypothesis to both methodologists and patient investigators. Methodologists found that 1 of 50 conveyed a study purpose. Patient investigators found that 11 of 50 conveyed a study purpose, 7 accurately and 4 inaccurately compared with the reference standard. Seventy-one of 72 noninferiority trial protocols or SAPs provided no rationale for selection of noninferiority hypothesis. None provided a clinical rationale for the chosen amount of decreased efficacy. Conclusions and Relevance Patients were not accurately informed of study purpose, which raises questions regarding the ethics of informed consent in antibiotic trials. Noninferiority and superiority trials entail different benefit-harm trade-offs that must be conveyed for ethical informed consent.

Donald Trump’s war with the media shows why medical journalists must acknowledge that not everyone thinks as they do, says Peter Doshi

One group’s efforts to monitor misreporting of outcomes has irritated several medical journals, which argue that the differences discovered are not clinically important. So how seriously should we be taking it? Peter Doshi reports Glimpses inside SmithKline Beecham’s secret clinical trials programme for the antidepressant paroxetine began in the early 2000s. Amid a growing storm over the safety of selective serotonin reuptake inhibitors for children, a leaked memo revealed by the BBC’s Panorama programme1 depicted a company trying to manage the unfavourable results of two important trials. “It would be commercially unacceptable to include a statement that efficacy had not been demonstrated, as this would undermine the profile of paroxetine,” the memo read. A reviewer for the US Food and Drug Administration considered both trials as “failed.”2 But then there was the public face of the data. One of the two trials–Study 329–was published in the peer reviewed literature.3 The manufacturer told its sales representatives that the “landmark study … demonstrates REMARKABLE efficacy and safety.”4 Study 329 has become a classic example of what is known as outcome reporting bias, in which trial authors selectively present trial results leading, almost inevitably, to a rosier picture than would have occurred had the trial been reported according to the original protocol. The data showed no difference between paroxetine and placebo for all eight of the originally specified outcomes of interest, and an increase in harms.5 6 7 Yet the 2001 trial publication3 reported on four outcomes not specified in the protocol (all of which had statistically significant differences) and concluded that paroxetine was “generally well tolerated and effective.” Put simply, the goalposts set when the trial commenced had moved by the time the trial was reported. Some may have suspected that Study 329 was an anomaly. …

Background Clinical trial (and other) data from the European Medicines Agency (EMA) offers the best available opportunity to address the extensive reporting bias in pharmaceutical trial literature. Data are requested via freedom of information requests, but 5 years on, little is known about how the system is working. Methods Case series of 12 requests for regulatory data (clinical study reports and other regulatory data) relating to 29 different compounds. We logged start and end dates for correspondence with and data releases from the EMA, the need for additional correspondence and appeal of initial negative decisions, and inspected data releases for redaction. We measured: time from initial request to first substantive response from the EMA, to final decision from the EMA (in case of appeal), to initial receipt of documents, and to completion of request; number of data transmission batches generated; number of pages received for each request; average number of pages per batch over time (for releases in multiple batches); judgment as to whether the request was satisfied. Results We found great variability in time to receive an initial decision from the EMA (1 to 13 weeks). Additional correspondence with the EMA was necessary in 10 of 12 requests. Four of 12 were initially refused but 3 of 4 were allowed on appeal after 3 to 33 additional weeks. One request was denied despite appeal. Time to final decision was 1 to 43 weeks. We received data for 11 of 12 requests in 98 batches. While two requests remain outstanding as at June 2015 the remaining nine requests took a median 43 weeks to completion (range: 17 to 186 weeks). Despite redaction in 10 of 11 releases (mainly of researcher and participant identifying information), 8 requested were wholly satisfied. Conclusions The EMA is the only regulator in the world that is routinely releasing original clinical trial data, but release can take considerable time to occur and often only after a lengthy correspondence. Given its importance for research and significance for transparency we suggest ways in which the process could be made more efficient. Electronic supplementary material The online version of this article (doi:10.1186/s13063-016-1194-7) contains supplementary material, which is available to authorized users.

In an opinion piece in the New York Times , the spouse of a lung cancer patient called a pharmaceutical company “shameful” for airing a television commercial that “prey[ed] upon the fears and waning hopes of terminal cancer patients.”[1][1] The commercial trumpeted “a chance to live longer

Eight years after the pandemic influenza outbreak, a lawsuit alleging that GlaxoSmithKline’s Pandemrix vaccine caused narcolepsy has unearthed internal reports suggesting problems with the vaccine’s safety. Peter Doshi asks what this means for the future of transparency during public health emergencies

The European Medicines Agency has scaled back its landmark clinical trial data policies, the drug regulator has announced. Policy 0043, which granted public access to documents in the regulators’ archives, including clinical trial data, is now limited to EU citizens only, and work related to publishing new data packages under its clinical data publication policy (policy 0070) has been suspended. The agency said that the cutbacks were a response to an “excessive workload”1 and were part of its planning amid substantial staff loss as it prepared for Brexit.2 Transparency advocates have described it as a blow to critical transparency efforts with potential patient safety ramifications. “Deaths and other serious harms are much under-reported in published trials, and we therefore need unhindered and immediate access to clinical study reports and other relevant documentation that the EMA holds,” said Peter Gotzsche, director of the Nordic Cochrane Centre. Gotzsche is credited with “opening up” the EMA after his successful appeal to the European ombudsman over a denied request for data a decade ago.3 He said that the EMA’s announcements were “very …

The correct title of the article [1] should be “Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol”.

The US Food and Drug Administration (FDA) has announced it will disclose clinical study reports (CSRs) in a pilot programme to begin this month.12 Participation in the pilot will be voluntary for sponsors, include up to nine recently approved drug applications, and is limited to CSRs for the key “pivotal” trials that underpin drug approval. Declaring that “transparency can be a powerful tool for innovation,”3 FDA commissioner Scott Gottlieb also argued the public CSRs would help streamline the FDA’s evaluation of drug applications and “create a more efficient review process.”4 The announcement, made on 16 January at a transparency forum led by Johns Hopkins School of Public Health, thrusts the regulator into an activity senior officials have in recent years resisted, arguing that data sharing was an activity best left to the companies that own the data, not middlemen like the FDA. Historically, the agency’s position has been that laws such as the Trade Secrets Act prohibit it from releasing trial data,5 and there have been only a handful of cases where the FDA has released CSRs, sometimes in response to …

Academics who refused calls to share the data underlying their landmark trial that triggered the downfall of starches for fluid resuscitation have announced an “independent reanalysis,” which confirms their original findings.1 However, they have not made their data available to the wider scientific community—a key element of transparent research practice and what other academics have been calling for.23 The 7000 patient trial, CHEST (Crystalloid versus Hydroxyethyl Starch Trial),4 is one of the most important pieces of evidence concerning the effects of hydroxyethyl starch solutions for increasing intravascular volume in patients in intensive care. Published in the New England Journal of Medicine in 2012, CHEST reported that the hydroxyethyl starch investigated, Voluven, was no different than saline in terms of mortality but led to greater use of renal replacement therapy. The trial helped persuade drug regulators in Europe and the US to issue safety warnings in 2013. In February the US health watchdog Public Citizen sent a petition to the Food and Drug Administration calling for hydroxyethyl starch intravenous solutions to be removed from the market,5 a move supported by CHEST’s lead author, John Myburgh.6 But the global healthcare company Fresenius Kabi, which makes Voluven and was a major funder (€3.5m (£3.15m; $4m)) of the trial, has long contended that the study was improperly reported.2 Before the trial the company took the unusual step of signing away all rights to the data, a decision it later came to regret. In 2014 it sought the assistance of The BMJ to intervene with the CHEST investigators. It then asked the Yale Open Data Access Project (YODA) to serve as an independent intermediary between the CHEST investigators and a reanalysis team to be determined, but the academic investigators rejected YODA’s offer.2 The latest “independent analysis” is …

Peter Doshi asks why no one seems to have noticed that the website isn’t working properly and what it says about the state of pharmacovigilance

Why they must help us improve the evidence base In 1990, a research scientist at the US Food and Drug Administration wrote in a letter to the New England Journal of Medicine ( NEJM ), “I review many clinical-trial protocols. The results of some of these trials subsequently appear in the medical literature. On occasion, the published description of the study may differ from the prospective protocol in important aspects of study design or statistical analysis—eg, study size, clinical end-points, and statistical tests used. The potential for misuse … is tremendous.”1 These comments were eerily prophetic: subsequent years brought the scandals of rofecoxib,23 celecoxib,4 paroxetine,56 and oseltamivir.7 The comments also suggest that each of these scandals could have largely been averted if journals had taken better account of what regulators—particularly the FDA—had to say about the relevant trials. We agree. Long before NEJM issued an expression of concern about data on harms missing from its report of the VIGOR trial of rofecoxib, those same data were available on the FDA’s website.2 JAMA reported CLASS as a six month trial …

A little noticed FDA report reveals internal opposition to Trump plans to speed drugs to market. Peter Doshi looks at its arguments

Objectives To investigate the practice of post-marketing studies in Germany during a three year period and to evaluate whether these trials meet the aims specified in the German Medicinal Products Act. Design Survey of notifications submitted to German regulatory agencies before post-marketing studies were carried out, 2008-10. Setting Notifications obtained through freedom of information requests to the three authorities responsible for registering post-marketing studies in Germany. Main outcome measures Descriptive statistics of post-marketing studies, including the products under study, intended number of patients, intended number of participating physicians, proposed remunerations, study plan and protocol, and availability of associated scientific publications and reports on adverse drug reactions. Results Information was obtained from 558 studies, with a median of 600 (mean 2331, range 2-75 000) patients and 63 (270, 0-7000) participating physicians per study. The median remuneration to physicians per patient was €200 (€441, €0-€7280) (£170, £0-£6200; $215, $0-$7820), with a total remuneration cost of more than €217m for 558 studies registered over the three year period. The median remuneration per participating physician per study was €2000 (mean €19 424), ranging from €0 to €2 080 000. There was a broad range of drugs and non-drug products, of which only a third represented recently approved drugs. In many notifications, data, information, and results were, by contract, strictly confidential and the sole property of the respective sponsor. No single adverse drug reaction report could be identified from any of the 558 post-marketing studies. Less than 1% of studies could be verified as published in scientific journals. Conclusions Post-marketing studies are not improving drug safety surveillance. Sample sizes are generally too small to allow the detection of rare adverse drug reactions, and many participating physicians are strictly obliged to maintain confidentiality towards the sponsor. High remuneration and strict confidentiality clauses in these studies could influence the physicians’ reporting behaviours of adverse drug reactions. BMJ2017;356:j337

Regulatory agencies such as Health Canada and the US Food and Drug Administration (FDA) have to date treated submissions of clinical trial data as confidential business information. Even for substances for which trials have been published, regulatory agencies possess substantial unpublished data

After publication of my article,1 the New England Journal of Medicine ( NEJM ) published a correction to the 2012 CHEST trial of hydroxyethyl starch versus colloids.2 3 A new P value of 0.006 was provided for the comparison of treatment related adverse events between hydroxyethyl starch (HES) and saline groups (180/3871 (4.6%) v 95/2870 (3.3%)). Before the correction, this was reported as 180/3416 (5.3%) …

Hydroxyethyl starch solutions for fluid resuscitation fell from grace in 2013 after European and American regulators issued severe warnings about their safety. But the academic investigators that led a landmark trial that helped precipitate this downfall are refusing to share their data. Is this acceptable? Peter Doshi reports

Peter Doshi and colleagues describe their experience trying and failing to access clinical study reports from the manufacturer of Tamiflu and challenge industry to defend their current position of RCT data secrecy.

BACKGROUND: Neuraminidase inhibitors (NIs) are stockpiled and recommended by public health agencies for treating and preventing seasonal and pandemic influenza. They are used clinically worldwide. OBJECTIVES: To describe the potential benefits and harms of NIs for influenza in all age groups by reviewing all clinical study reports of published and unpublished randomised, placebo‐controlled trials and regulatory comments. SEARCH METHODS: We searched trial registries, electronic databases (to 22 July 2013) and regulatory archives, and corresponded with manufacturers to identify all trials. We also requested clinical study reports. We focused on the primary data sources of manufacturers but we checked that there were no published randomised controlled trials (RCTs) from non‐manufacturer sources by running electronic searches in the following databases: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, MEDLINE (Ovid), EMBASE, Embase.com, PubMed (not MEDLINE), the Database of Reviews of Effects, the NHS Economic Evaluation Database and the Health Economic Evaluations Database. SELECTION CRITERIA: Randomised, placebo‐controlled trials on adults and children with confirmed or suspected exposure to naturally occurring influenza. DATA COLLECTION AND ANALYSIS: We extracted clinical study reports and assessed risk of bias using purpose‐built instruments. We analysed the effects of zanamivir and oseltamivir on time to first alleviation of symptoms, influenza outcomes, complications, hospitalisations and adverse events in the intention‐to‐treat (ITT) population. All trials were sponsored by the manufacturers. MAIN RESULTS: We obtained 107 clinical study reports from the European Medicines Agency (EMA), GlaxoSmithKline and Roche. We accessed comments by the US Food and Drug Administration (FDA), EMA and Japanese regulator. We included 53 trials in Stage 1 (a judgement of appropriate study design) and 46 in Stage 2 (formal analysis), including 20 oseltamivir (9623 participants) and 26 zanamivir trials (14,628 participants). Inadequate reporting put most of the zanamivir studies and half of the oseltamivir studies at a high risk of selection bias. There were inadequate measures in place to protect 11 studies of oseltamivir from performance bias due to non‐identical presentation of placebo. Attrition bias was high across the oseltamivir studies and there was also evidence of selective reporting for both the zanamivir and oseltamivir studies. The placebo interventions in both sets of trials may have contained active substances. Time to first symptom alleviation. For the treatment of adults, oseltamivir reduced the time to first alleviation of symptoms by 16.8 hours (95% confidence interval (CI) 8.4 to 25.1 hours, P < 0.0001). This represents a reduction in the time to first alleviation of symptoms from 7 to 6.3 days. There was no effect in asthmatic children, but in otherwise healthy children there was (reduction by a mean difference of 29 hours, 95% CI 12 to 47 hours, P = 0.001). Zanamivir reduced the time to first alleviation of symptoms in adults by 0.60 days (95% CI 0.39 to 0.81 days, P < 0.00001), equating to a reduction in the mean duration of symptoms from 6.6 to 6.0 days. The effect in children was not significant. In subgroup analysis we found no evidence of a difference in treatment effect for zanamivir on time to first alleviation of symptoms in adults in the influenza‐infected and non‐influenza‐infected subgroups (P = 0.53). Hospitalisations. Treatment of adults with oseltamivir had no significant effect on hospitalisations: risk difference (RD) 0.15% (95% CI ‐0.78 to 0.91). There was also no significant effect in children or in prophylaxis. Zanamivir hospitalisation data were unreported. Serious influenza complications or those leading to study withdrawal. In adult treatment trials, oseltamivir did not significantly reduce those complications classified as serious or those which led to study withdrawal (RD 0.07%, 95% CI ‐0.78 to 0.44), nor in child treatment trials; neither did zanamivir in the treatment of adults or in prophylaxis. There were insufficient events to compare this outcome for oseltamivir in prophylaxis or zanamivir in the treatment of children. Pneumonia. Oseltamivir significantly reduced self reported, investigator‐mediated, unverified pneumonia (RD 1.00%, 95% CI 0.22 to 1.49); number needed to treat to benefit (NNTB) = 100 (95% CI 67 to 451) in the treated population. The effect was not significant in the five trials that used a more detailed diagnostic form for pneumonia. There were no definitions of pneumonia (or other complications) in any trial. No oseltamivir treatment studies reported effects on radiologically confirmed pneumonia. There was no significant effect on unverified pneumonia in children. There was no significant effect of zanamivir on either self reported or radiologically confirmed pneumonia. In prophylaxis, zanamivir significantly reduced the risk of self reported, investigator‐mediated, unverified pneumonia in adults (RD 0.32%, 95% CI 0.09 to 0.41); NNTB = 311 (95% CI 244 to 1086), but not oseltamivir. Bronchitis, sinusitis and otitis media. Zanamivir significantly reduced the risk of bronchitis in adult treatment trials (RD 1.80%, 95% CI 0.65 to 2.80); NNTB = 56 (36 to 155), but not oseltamivir. Neither NI significantly reduced the risk of otitis media and sinusitis in both adults and children. Harms of treatment. Oseltamivir in the treatment of adults increased the risk of nausea (RD 3.66%, 95% CI 0.90 to 7.39); number needed to treat to harm (NNTH) = 28 (95% CI 14 to 112) and vomiting (RD 4.56%, 95% CI 2.39 to 7.58); NNTH = 22 (14 to 42). The proportion of participants with four‐fold increases in antibody titre was significantly lower in the treated group compared to the control group (RR 0.92, 95% CI 0.86 to 0.97, I(2) statistic = 0%) (5% absolute difference between arms). Oseltamivir significantly decreased the risk of diarrhoea (RD 2.33%, 95% CI 0.14 to 3.81); NNTB = 43 (95% CI 27 to 709) and cardiac events (RD 0.68%, 95% CI 0.04 to 1.0); NNTB = 148 (101 to 2509) compared to placebo during the on‐treatment period. There was a dose‐response effect on psychiatric events in the two oseltamivir "pivotal" treatment trials, WV15670 and WV15671, at 150 mg (standard dose) and 300 mg daily (high dose) (P = 0.038). In the treatment of children, oseltamivir induced vomiting (RD 5.34%, 95% CI 1.75 to 10.29); NNTH = 19 (95% CI 10 to 57). There was a significantly lower proportion of children on oseltamivir with a four‐fold increase in antibodies (RR 0.90, 95% CI 0.80 to 1.00, I(2) = 0%). Prophylaxis. In prophylaxis trials, oseltamivir and zanamivir reduced the risk of symptomatic influenza in individuals (oseltamivir: RD 3.05% (95% CI 1.83 to 3.88); NNTB = 33 (26 to 55); zanamivir: RD 1.98% (95% CI 0.98 to 2.54); NNTB = 51 (40 to 103)) and in households (oseltamivir: RD 13.6% (95% CI 9.52 to 15.47); NNTB = 7 (6 to 11); zanamivir: RD 14.84% (95% CI 12.18 to 16.55); NNTB = 7 (7 to 9)). There was no significant effect on asymptomatic influenza (oseltamivir: RR 1.14 (95% CI 0.39 to 3.33); zanamivir: RR 0.97 (95% CI 0.76 to 1.24)). Non‐influenza, influenza‐like illness could not be assessed due to data not being fully reported. In oseltamivir prophylaxis studies, psychiatric adverse events were increased in the combined on‐ and off‐treatment periods (RD 1.06%, 95% CI 0.07 to 2.76); NNTH = 94 (95% CI 36 to 1538) in the study treatment population. Oseltamivir increased the risk of headaches whilst on treatment (RD 3.15%, 95% CI 0.88 to 5.78); NNTH = 32 (95% CI 18 to 115), renal events whilst on treatment (RD 0.67%, 95% CI ‐2.93 to 0.01); NNTH = 150 (NNTH 35 to NNTB > 1000) and nausea whilst on treatment (RD 4.15%, 95% CI 0.86 to 9.51); NNTH = 25 (95% CI 11 to 116). AUTHORS' CONCLUSIONS: Oseltamivir and zanamivir have small, non‐specific effects on reducing the time to alleviation of influenza symptoms in adults, but not in asthmatic children. Using either drug as prophylaxis reduces the risk of developing symptomatic influenza. Treatment trials with oseltamivir or zanamivir do not settle the question of whether the complications of influenza (such as pneumonia) are reduced, because of a lack of diagnostic definitions. The use of oseltamivir increases the risk of adverse effects, such as nausea, vomiting, psychiatric effects and renal events in adults and vomiting in children. The lower bioavailability may explain the lower toxicity of zanamivir compared to oseltamivir. The balance between benefits and harms should be considered when making decisions about use of both NIs for either the prophylaxis or treatment of influenza. The influenza virus‐specific mechanism of action proposed by the producers does not fit the clinical evidence.

Will outsourcing determination of antibiotic effectiveness allow drug companies to influence prescribing? Peter Doshi reports

Objective To explore the structure and content of a non-random sample of clinical study reports (CSRs) to guide clinicians and systematic reviewers. Search strategy We searched public sources and lodged Freedom of Information requests for previously confidential CSRs primarily written by the industry for regulators. Selection criteria CSRs reporting sufficient information for extraction (‘adequate’). Primary outcome measures Presence and length of essential elements of trial design and reporting and compression factor (ratio of page length for CSRs compared to its published counterpart in a scientific journal). Data extraction Data were extracted on standard forms and crosschecked for accuracy. Results We assembled a population of 78 CSRs (covering 90 randomised controlled trials; 144 610 pages total) dated 1991–2011 of 14 pharmaceuticals. Report synopses had a median length of 5 pages, efficacy evaluation 13.5 pages, safety evaluation 17 pages, attached tables 337 pages, trial protocol 62 pages, statistical analysis plan 15 pages and individual efficacy and safety listings had a median length of 447 and 109.5 pages, respectively. While 16 (21%) of CSRs contained completed case report forms, these were accessible to us in only one case (765 pages representing 16 individuals). Compression factors ranged between 1 and 8805. Conclusions Clinical study reports represent a hitherto mostly hidden and untapped source of detailed and exhaustive data on each trial. They should be consulted by independent parties interested in a detailed record of a clinical trial, and should form the basic unit for evidence synthesis as their use is likely to minimise the problem of reporting bias. We cannot say whether our sample is representative and whether our conclusions are generalisable to an undefined and undefinable population of CSRs.

In recent years there has been much debate and controversy surrounding the efficacy and safety of neuraminidase inhibitors for influenza, in part because the data underlying certain efficacy claims were not available for independent scrutiny. In 2014, a Cochrane review was published, based exclusively on an almost complete set of clinical study reports and other regulatory documents. Clinical study reports can run to thousands of pages, providing an extensive amount of information on the planning, conduct and results of each trial. After a protracted campaign to obtain the reports, the manufacturers of the medications provided them unconditionally. The review authors subsequently published the underlying documents simultaneously with the Cochrane review, endorsing the concept of open science. In the following commentary, the background to and results of this review are summarized and put into clinical context.