Search

Shortcomings in the rigour and reproducibility of research have become well-known issues and persist despite repeated calls for improvement. A coordinated effort among researchers, institutions, funders, publishers, learned societies, and regulators may be the most effective way of tackling these issues. The UK Reproducibility Network (UKRN) has fostered collaboration across various stakeholders in research and are creating the infrastructure necessary to advance rigorous and reproducible research practices across the United Kingdom. Other Reproducibility Networks, modelled on UKRN, are now emerging in other countries. Canada could benefit from a comparable network to unify the voices around research quality and maximize the value of Canadian research.

The Strategy for Patient Oriented Research (SPOR) Evidence Alliance is a research initiative in Canada whose mission is to promote the synthesis, dissemination, and integration of research results into health care and public health decision-making and clinical practice. The aim of this paper is to (i) outline the governance and committee structure of the SPOR Evidence Alliance, (ii) outline the procedures for patient and health system decision-maker engagement, and (iii) present the capacity-building strategy for governance members. The governance structure includes the following six standing committees: the International Advisory Committee, Steering Committee, Executive Committee, Knowledge Translation Committee, Partnerships Committee, and Training and Capacity Development Committee. The guiding principles embrace inclusiveness, support, mutual respect, transparency, and co-building. There are currently 64 committee members across the six committees, 13 patient and public partners, 8 health system decision-makers, 7 research trainees, and 36 researchers. A multi-disciplinary and diverse group of people in Canada are represented from all regions and at various levels of training in knowledge generation, exchange, and translation. This collaborative model makes the SPOR Evidence Alliance strong and sustainable by leveraging the knowledge, lived experiences, expertise, skills, and networks among its 342 members and 12 principal investigators.

In 2009, the Canadian Institutes of Health Research, Health Canada, and other stakeholders established the Drug Safety and Effectiveness Network (DSEN) to address the paucity of information on drug safety and effectiveness in real-world settings. This unique network invited knowledge users (e.g., policy makers) to submit queries to be answered by relevant research teams. The research teams were launched via open calls for team grants focused in relevant methodologic areas. We describe the development and implementation of one of these collaborating centres, the Methods and Application Group for Indirect Comparisons (MAGIC). MAGIC was created to provide high-quality knowledge synthesis including network meta-analysis to meet knowledge user needs. Since 2011, MAGIC responded to 54% of queries submitted to DSEN. In the past 5 years, MAGIC produced 26 reports and 49 publications. It led to 15 trainees who entered industry, academia, and government. More than 10 000 people participated in courses delivered by MAGIC team members. Most importantly, MAGIC knowledge syntheses influenced practice and policy (e.g., use of biosimilars for patients with diabetes and use of smallpox vaccinations in people with contraindications) provincially, nationally, and internationally.

BackgroundScholarly journals play a key role in the dissemination of research findings. However, little focus is given to the process of establishing new, credible journals and the obstacles faced in achieving this. This scoping review aimed to identify and describe existing recommendations for starting a biomedical scholarly journal.MethodsWe searched five bibliographic databases: OVID Medline + Medline in Process, Embase Classic + Embase, ERIC, APA PsycINFO, and Web of Science on January 14, 2022. A related grey literature search was conducted on March 19, 2022. Eligible sources were those published in English in any year, of any format, and that described guidance for starting a biomedical journal. Titles and abstracts of obtained sources were screened. We extracted descriptive characteristics including author name, year and country of publication, journal name, and source type, and any recommendations from the included sources discussing guidance for starting a biomedical journal. These recommendations were categorized and thematically grouped.ResultsA total of 5626 unique sources were obtained. Thirty-three sources met our inclusion criteria. Most sources were blog posts (10/33; 30.30%), and only 10 sources were supported by evidence. We extracted 51 unique recommendations from these 33 sources, which we thematically classified into nine themes which were: journal operations, editorial review processes, peer review processes, open access publishing, copyediting/typesetting, production, archiving/indexing/metrics, marketing/promotion, and funding.ConclusionsThere is little formal guidance regarding how to start a scholarly journal. The development of an evidence-based guideline may help uphold scholarly publishing quality, provide insight into obstacles new journals will face, and equip novice publishers with the tools to meet best practices.

The state of open science needs to be monitored to track changes over time and identify areas to create interventions to drive improvements. In order to monitor open science practices, they first need to be well defined and operationalized. To reach consensus on what open science practices to monitor at biomedical research institutions, we conducted a modified 3-round Delphi study. Participants were research administrators, researchers, specialists in dedicated open science roles, and librarians. In rounds 1 and 2, participants completed an online survey evaluating a set of potential open science practices, and for round 3, we hosted two half-day virtual meetings to discuss and vote on items that had not reached consensus. Ultimately, participants reached consensus on 19 open science practices. This core set of open science practices will form the foundation for institutional dashboards and may also be of value for the development of policy, education, and interventions.

Abstract Background Problems continue to exist with the reporting quality and risk of bias in search methods and strategies in systematic reviews and related review types. Peer reviewers who are not familiar with what is required to transparently and fully report a search may not be prepared to review the search components of systematic reviews, nor may they know what is likely to introduce bias into a search. Librarians and information specialists, who have expertise in searching, may offer specialized knowledge that would help improve systematic review search reporting and lessen risk of bias, but they are underutilized as methodological peer reviewers. Methods This study will evaluate the effect of adding librarians and information specialists as methodological peer reviewers on the quality of search reporting and risk of bias in systematic review searches. The study will be a pragmatic randomized controlled trial using 150 systematic review manuscripts submitted to BMJ and BMJ Open as the unit of randomization. Manuscripts that report on completed systematic reviews and related review types and have been sent for peer review are eligible. For each manuscript randomized to the intervention, a librarian/information specialist will be invited as an additional peer reviewer using standard practices for each journal. First revision manuscripts will be assessed in duplicate for reporting quality and risk of bias, using adherence to 4 items from PRISMA-S and assessors’ judgements on 4 signaling questions from ROBIS Domain 2, respectively. Identifying information from the manuscripts will be removed prior to assessment. Discussion The primary outcomes for this study are quality of reporting as indicated by differences in the proportion of adequately reported searches in first revision manuscripts between intervention and control groups and risk of bias as indicated by differences in the proportions of first revision manuscripts with high, low, and unclear bias. If the intervention demonstrates an effect on search reporting or bias, this may indicate a need for journal editors to work with librarians and information specialists as methodological peer reviewers. Trial registration Open Science Framework. Registered on June 17, 2021, at https://doi.org/10.17605/OSF.IO/W4CK2 .

Highlights Omission of sex and gender considerations is a recurring deficiency in research design and reporting Integration of sex and gender considerations in health research reporting guidelines is very low Three criteria were used to assess correct use of sex and gender concepts Only one reporting guideline met the three criteria A call to action is made to address these deficiencies

Abstract Background Convalescent plasma has been widely used to treat COVID-19 and is under investigation in numerous randomized clinical trials, but results are publicly available only for a small number of trials. The objective of this study was to assess the benefits of convalescent plasma treatment compared to placebo or no treatment and all-cause mortality in patients with COVID-19, using data from all available randomized clinical trials, including unpublished and ongoing trials (Open Science Framework, https://doi.org/10.17605/OSF.IO/GEHFX ). Methods In this collaborative systematic review and meta-analysis, clinical trial registries (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform), the Cochrane COVID-19 register, the LOVE database, and PubMed were searched until April 8, 2021. Investigators of trials registered by March 1, 2021, without published results were contacted via email. Eligible were ongoing, discontinued and completed randomized clinical trials that compared convalescent plasma with placebo or no treatment in COVID-19 patients, regardless of setting or treatment schedule. Aggregated mortality data were extracted from publications or provided by investigators of unpublished trials and combined using the Hartung–Knapp–Sidik–Jonkman random effects model. We investigated the contribution of unpublished trials to the overall evidence. Results A total of 16,477 patients were included in 33 trials (20 unpublished with 3190 patients, 13 published with 13,287 patients). 32 trials enrolled only hospitalized patients (including 3 with only intensive care unit patients). Risk of bias was low for 29/33 trials. Of 8495 patients who received convalescent plasma, 1997 died (23%), and of 7982 control patients, 1952 died (24%). The combined risk ratio for all-cause mortality was 0.97 (95% confidence interval: 0.92; 1.02) with between-study heterogeneity not beyond chance (I2 = 0%). The RECOVERY trial had 69.8% and the unpublished evidence 25.3% of the weight in the meta-analysis. Conclusions Convalescent plasma treatment of patients with COVID-19 did not reduce all-cause mortality. These results provide strong evidence that convalescent plasma treatment for patients with COVID-19 should not be used outside of randomized trials. Evidence synthesis from collaborations among trial investigators can inform both evidence generation and evidence application in patient care.

Abstract Cell‐based therapies hold promise to substantially curb complications from extreme preterm birth, the main cause of death in children below the age of 5 years. Exciting preclinical studies in experimental neonatal lung injury have provided the impetus for the initiation of early phase clinical trials in extreme preterm infants at risk of developing bronchopulmonary dysplasia. Clinical translation of promising therapies, however, is slow and often fails. In the adult population, results of clinical trials so far have not matched the enticing preclinical data. The neonatal field has experienced many hard‐earned lessons with the implementation of oxygen therapy or postnatal steroids. Here we briefly summarize the preclinical data that have permitted the initiation of early phase clinical trials of cell‐based therapies in extreme preterm infants and describe the INCuBAToR concept (Innovative Neonatal Cellular Therapy for Bronchopulmonary Dysplasia: Accelerating Translation of Research), an evidence‐based approach to mitigate the risk of translating advanced therapies into this vulnerable patient population. The INCuBAToR addresses several of the shortcomings at the preclinical and the clinical stage that usually contribute to the failure of clinical translation through (a) systematic reviews of preclinical and clinical studies, (b) integrated knowledge transfer through engaging important stakeholders early on, (c) early economic evaluation to determine if a novel therapy is viable, and (d) retrospective and prospective studies to define and test ideal eligibility criteria to optimize clinical trial design. The INCuBAToR concept can be applied to any novel therapy in order to enhance the likelihood of success of clinical translation in a timely, transparent, rigorous, and evidence‐based fashion.

Plain English summary In academia, authorship on a research publication is a central means to obtain credit for one’s contribution to a research project. In order to guide authorship decision making and facilitate transparent processes, the International Committee of Medical Journal Editors (ICMJE) has produced recommendations for authorship. These recommendations are widely adopted by hundreds of medical journals. However, there is no research on whether the ICJME’s recommendations for authorship appropriately recognize the distinct contributions of patient partners to research. This survey study asked Editors-in-Chief about their perceptions of patient partners as authors and on the suitability of the ICMJE authorship criteria. We find that a 30.8%of medical journal Editors-in-chief do not view the inclusion of patient partners as authors on manuscripts as appropriate. Editors-in-chief reported wide ranging views on whether the ICMJE criteria for authorship were appropriate for patient partners. The implication of this work is that there is a need for education and for consensus building within the biomedical community to establish processes that will facilitate patient partners equitable inclusion in research, including on research outputs like publications.

Abstract Background Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. Methods The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). Discussion The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

Abstract Substantial COVID-19 research investment has been allocated to randomized clinical trials (RCTs) on hydroxychloroquine/chloroquine, which currently face recruitment challenges or early discontinuation. We aim to estimate the effects of hydroxychloroquine and chloroquine on survival in COVID-19 from all currently available RCT evidence, published and unpublished. We present a rapid meta-analysis of ongoing, completed, or discontinued RCTs on hydroxychloroquine or chloroquine treatment for any COVID-19 patients (protocol: https://osf.io/QESV4/ ). We systematically identified unpublished RCTs (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, Cochrane COVID-registry up to June 11, 2020), and published RCTs (PubMed, medRxiv and bioRxiv up to October 16, 2020). All-cause mortality has been extracted (publications/preprints) or requested from investigators and combined in random-effects meta-analyses, calculating odds ratios (ORs) with 95% confidence intervals (CIs), separately for hydroxychloroquine and chloroquine. Prespecified subgroup analyses include patient setting, diagnostic confirmation, control type, and publication status. Sixty-three trials were potentially eligible. We included 14 unpublished trials (1308 patients) and 14 publications/preprints (9011 patients). Results for hydroxychloroquine are dominated by RECOVERY and WHO SOLIDARITY, two highly pragmatic trials, which employed relatively high doses and included 4716 and 1853 patients, respectively (67% of the total sample size). The combined OR on all-cause mortality for hydroxychloroquine is 1.11 (95% CI: 1.02, 1.20; I² = 0%; 26 trials; 10,012 patients) and for chloroquine 1.77 (95%CI: 0.15, 21.13, I² = 0%; 4 trials; 307 patients). We identified no subgroup effects. We found that treatment with hydroxychloroquine is associated with increased mortality in COVID-19 patients, and there is no benefit of chloroquine. Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities.

Abstract Background The application of artificial intelligence (AI) in healthcare is an area of immense interest. The high profile of ‘AI in health’ means that there are unusually strong drivers to accelerate the introduction and implementation of innovative AI interventions, which may not be supported by the available evidence, and for which the usual systems of appraisal may not yet be sufficient. Main text We are beginning to see the emergence of randomised clinical trials evaluating AI interventions in real-world settings. It is imperative that these studies are conducted and reported to the highest standards to enable effective evaluation because they will potentially be a key part of the evidence that is used when deciding whether an AI intervention is sufficiently safe and effective to be approved and commissioned. Minimum reporting guidelines for clinical trial protocols and reports have been instrumental in improving the quality of clinical trials and promoting completeness and transparency of reporting for the evaluation of new health interventions. The current guidelines—SPIRIT and CONSORT—are suited to traditional health interventions but research has revealed that they do not adequately address potential sources of bias specific to AI systems. Examples of elements that require specific reporting include algorithm version and the procedure for acquiring input data. In response, the SPIRIT-AI and CONSORT-AI guidelines were developed by a multidisciplinary group of international experts using a consensus building methodological process. The extensions include a number of new items that should be reported in addition to the core items. Each item, where possible, was informed by challenges identified in existing studies of AI systems in health settings. Conclusion The SPIRIT-AI and CONSORT-AI guidelines provide the first international standards for clinical trials of AI systems. The guidelines are designed to ensure complete and transparent reporting of clinical trial protocols and reports involving AI interventions and have the potential to improve the quality of these clinical trials through improvements in their design and delivery. Their use will help to efficiently identify the safest and most effective AI interventions and commission them with confidence for the benefit of patients and the public.

Abstract Background Major depressive disorder is common, debilitating, and affects feelings, thoughts, mood, and behaviors. Childhood and adolescence are critical periods for the development of depression and adolescence is marked by an increased incidence of mental health disorders. This protocol outlines the planned scope and methods for a systematic review update that will evaluate the benefits and harms of screening for depression in children and adolescents. Methods This review will update a previously published systematic review by Roseman and colleagues. Eligible studies are randomized controlled trials (RCTs) assessing formal screening in primary care to identify children or adolescents not already self-reporting symptoms of, diagnosed with, or treated for depression. If no or only a single RCT is available, we will consider controlled studies without random assignment. Studies of participants with characteristics associated with an elevated risk of depression will be analyzed separately. Outcomes of interest are symptoms of depression, classification of major depressive disorder based on a validated diagnostic interview, suicidality, health-related quality of life, social function, impact on lifestyle behavior (e.g., substance use, school performance, lost time at work, or school), false-positive results, overdiagnosis, overtreatment, labeling, and other harms such as those arising from treatment. We will search MEDLINE, Embase, PsycINFO, CINAHL, the Cochrane Library, and grey literature sources. Two reviewers will independently screen the titles and abstracts using the liberal accelerated method. Full-text screening will be performed independently by two reviewers using pre-specified eligibility criteria. Data extraction and risk of bias assessments will be performed independently by two reviewers. Pre-planned analyses, including subgroup and sensitivity analyses, are detailed within this protocol. Two independent reviewers will assess and finalize through consensus the certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, and prepare GRADE evidence profiles and summary of findings tables for each outcome of interest. Discussion The systematic review will provide a current state of the evidence of benefits and harms of depression screening in children and adolescents. These findings will be used by the Canadian Task Force on Preventive Health Care to inform the development of recommendations on depression screening. Systematic review registration PROSPERO CRD42020150373

Abstract Background Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse, and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration. Methods The checklist was developed using a 3-stage Delphi survey process, followed by a consensus conference and public review process. Results The final checklist includes 16 reporting items, each of which is detailed with exemplar reporting and rationale. Conclusions The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and therefore reproducible.

Background: The objective of this study was to determine the presence of a set of prespecified criteria used to assess scientists for promotion and tenure within faculties of medicine among the U15 Group of Canadian Research Universities. Methods: Each faculty guideline for assessing promotion and tenure was reviewed and the presence of five traditional (peer-reviewed publications, authorship order, journal impact factor, grant funding, and national/international reputation) and seven nontraditional (citations, data sharing, publishing in open access mediums, accommodating leaves, alternative ways for sharing research, registering research, using reporting guidelines) criteria were collected by two reviewers. Results: Among the U15 institutions, four of five traditional criteria (80.0%) were present in at least one promotion guideline, whereas only three of seven nontraditional incentives (42.9%) were present in any promotion guidelines. When assessing full professors, there were a median of three traditional criteria listed, versus one nontraditional criterion. Conclusion: This study demonstrates that faculties of medicine among the U15 Group of Canadian Research Universities base assessments for promotion and tenure on traditional criteria. Some of these metrics may reinforce problematic practices in medical research. These faculties should consider incentivizing criteria that can enhance the quality of medical research.

The Canadian federal Tri-Agency Research Data Management Policy has recently been released. This will require Canadian universities and other research institutes to create and share strategic plans regarding data management and to equip their researchers with skills to complete data deposits. To help maximize the success of data sharing we outline five domains for research institutions to consider during implementation: training and education, paying for data sharing, audit and feedback, meta-science, and career advancement.

Summary: Background: The results and data availability of vaccine trials directly affect the decisions of healthcare providers, the public, and policymakers as to whether the vaccine should be applied. However, the reporting and data sharing level of COVID-19 vaccine studies are not clear. Methods: A cross-sectional study was conducted. A systematic search up to 9 May 2021 in 12 databases and an updated search to 6 July 2021 were conducted in the Cochrane Living Systematic Review and Network Meta-Analysis database to identify COVID-19 vaccine trials. The basic characteristics of included trials were summarized. The reporting level was assessed according to the CONSORT checklist. The data sharing level was assessed by open science practices. Types of incomplete reporting including protocol deviation, lack of primary outcomes clarity, and the omission of harms were analyzed. Findings: Finally, thirty-six COVID-19 vaccine articles reporting on 40 randomized controlled trials were included in this analysis. Based on the CONSORT checklist, the mean reporting score was 29.7 [95% confidence interval 28.7, 30.7]. Thirty-one articles (31/36, 86.1%) had data sharing statements, twenty-five articles (25/36, 69.4%) provided access to the source data. Twenty-seven articles (27/36, 75.0%) had protocol deviation, lack of primary outcomes clarity, or the omission of harms. Interpretation: The reporting and data sharing level of COVID-19 vaccine trials were not optimal. We hope that the reporting and data sharing of future trials will be improved. We recommend establishing a comprehensive, accurate data sharing system for future vaccine trials. Funding: This work was supported by the National Key R&D Program of China (2019YFC1710400; 2019YFC1710403).

Introduction Many predatory journals fail to follow best publication practices. Studies assessing the impact of predatory journals have focused on how these articles are cited in reputable academic journals. However, it is possible that research from predatory journals is cited beyond the academic literature in policy documents and guidelines. Given that research used to inform public policy or government guidelines has the potential for widespread impact, we will examine whether predatory journals have penetrated public policy.Methods and analysis This is a descriptive study with no hypothesis testing. Policy documents that cite work from the known predatory publisher OMICS will be downloaded from the Overton database. Overton collects policy documents from over 1200 sources worldwide. Policy documents will be evaluated to determine how the predatory journal article is used. We will also extract epidemiological details of the policy documents, including: who funded their development, the discipline the work is relevant to and the name of the organisations producing the policy. The record of scholarly citations of the identified predatory articles will also be examined. Findings will be reported with descriptive statistics using counts and percentages.Ethics and dissemination No ethical approval was required for this study since it does not involve human or animal research. Study findings will be discussed at workshops on journalology and predatory publishing and will be disseminated through preprint, peer-reviewed literature and conference presentations.

Abstract Background Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice. Methods We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere. Results Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read. Conclusions Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored. Protocol availability The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

Abstract Background Clinicians, patients, and policy-makers rely on published evidence from clinical trials to help inform decision-making. A lack of complete and transparent reporting of the investigated trial outcomes limits reproducibility of results and knowledge synthesis efforts, and contributes to outcome switching and other reporting biases. Outcome-specific extensions for the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT-Outcomes) and Consolidated Standards of Reporting Trials (CONSORT-Outcomes) reporting guidelines are under development to facilitate harmonized reporting of outcomes in trial protocols and reports. The aim of this review was to identify and synthesize existing guidance for trial outcome reporting to inform extension development. Methods We searched for documents published in the last 10 years that provided guidance on trial outcome reporting using: an electronic bibliographic database search (MEDLINE and the Cochrane Methodology Register); a grey literature search; and solicitation of colleagues using a snowballing approach. Two reviewers completed title and abstract screening, full-text screening, and data charting after training. Extracted trial outcome reporting guidance was compared with candidate reporting items to support, refute, or refine the items and to assess the need for the development of additional items. Results In total, 1758 trial outcome reporting recommendations were identified within 244 eligible documents. The majority of documents were published by academic journals (72%). Comparison of each recommendation with the initial list of 70 candidate items led to the development of an additional 62 items, producing 132 candidate items. The items encompassed outcome selection, definition, measurement, analysis, interpretation, and reporting of modifications between trial documents. The total number of documents supporting each candidate item ranged widely (median 5, range 0–84 documents per item), illustrating heterogeneity in the recommendations currently available for outcome reporting across a large and diverse sample of sources. Conclusions Outcome reporting guidance for clinical trial protocols and reports lacks consistency and is spread across a large number of sources that may be challenging to access and implement in practice. Evidence and consensus-based guidance, currently in development (SPIRIT-Outcomes and CONSORT-Outcomes), may help authors adequately describe trial outcomes in protocols and reports transparently and completely to help reduce avoidable research waste.

Abstract Background The increase in the number of predatory journals puts scholarly communication at risk. In order to guard against publication in predatory journals, authors may use checklists to help detect predatory journals. We believe there are a large number of such checklists yet it is uncertain whether these checklists contain similar content. We conducted a systematic review to identify checklists that help to detect potential predatory journals and examined and compared their content and measurement properties. Methods We searched MEDLINE, Embase, PsycINFO, ERIC, Web of Science and Library, and Information Science & Technology Abstracts (January 2012 to November 2018); university library websites (January 2019); and YouTube (January 2019). We identified sources with original checklists used to detect potential predatory journals published in English, French or Portuguese. Checklists were defined as having instructions in point form, bullet form, tabular format or listed items. We excluded checklists or guidance on recognizing “legitimate” or “trustworthy” journals. To assess risk of bias, we adapted five questions from A Checklist for Checklists tool a priori as no formal assessment tool exists for the type of review conducted. Results Of 1528 records screened, 93 met our inclusion criteria. The majority of included checklists to identify predatory journals were in English (n = 90, 97%), could be completed in fewer than five minutes (n = 68, 73%), included a mean of 11 items (range = 3 to 64) which were not weighted (n = 91, 98%), did not include qualitative guidance (n = 78, 84%), or quantitative guidance (n = 91, 98%), were not evidence-based (n = 90, 97%) and covered a mean of four of six thematic categories. Only three met our criteria for being evidence-based, i.e. scored three or more “yes” answers (low risk of bias) on the risk of bias tool. Conclusion There is a plethora of published checklists that may overwhelm authors looking to efficiently guard against publishing in predatory journals. The continued development of such checklists may be confusing and of limited benefit. The similarity in checklists could lead to the creation of one evidence-based tool serving authors from all disciplines.

Abstract Background When a journal receives a duplicate publication, the ability to identify the submitted work as previously published, and reject it, is an assay to publication ethics best practices. The aim of this study was to evaluate how three different types of journals, namely open access (OA) journals, subscription-based journals, and presumed predatory journals, responded to receiving a previously published manuscript for review. Methods We performed a quasi-experimental study in which we submitted a previously published article to a random sample of 602 biomedical journals, roughly 200 journals from each journal type sampled: OA journals, subscription-based journals, and presumed predatory journals. Three hundred and three journals received a Word version in manuscript format, while 299 journals received the formatted publisher’s PDF version of the published article. We then recorded responses to the submission received after approximately 1 month. Responses were reviewed, extracted, and coded in duplicate. Our primary outcome was the rate of rejection of the two types of submitted articles (PDF vs Word) within our three journal types. Results We received correspondence back from 308 (51.1%) journals within our study timeline (32 days); (N = 46 predatory journals, N = 127 OA journals, N = 135 subscription-based journals). Of the journals that responded, 153 received the Word version of the paper, while 155 received the PDF version. Four journals (1.3%) accepted our paper, 291 (94.5%) journals rejected the paper, and 13 (4.2%) requested a revision. A chi-square test looking at journal type, and submission type, was significant (χ 2 (4) = 23.50, p

Abstract Background The randomized controlled trial (RCT) is considered the gold standard study design to inform decisions about the effectiveness of interventions. However, a common limitation is inadequate reporting of the applicability of the intervention and trial results for people who are “socially disadvantaged” and this can affect policy-makers’ decisions. We previously developed a framework for identifying health-equity-relevant trials, along with a reporting guideline for transparent reporting. In this study, we provide a descriptive assessment of health-equity considerations in 200 randomly sampled equity-relevant trials. Methods We developed a search strategy to identify health-equity-relevant trials published between 2013 and 2015. We randomly sorted the 4316 records identified by the search and screened studies until 100 individually randomized (RCTs) and 100 cluster randomized controlled trials (CRTs) were identified. We developed and pilot-tested a data extraction form based on our initial work, to inform the development of our reporting guideline for equity-relevant randomized trials. Results In total, 39 trials (20%) were conducted in a low- and middle-income country and 157 trials (79%) in a high-income country focused on socially disadvantaged populations (78% CRTs, 79% RCTs). Seventy-four trials (37%) reported a subgroup analysis across a population characteristic associated with disadvantage (25% CRT, 49% RCTs), with 19% of included studies reporting subgroup analyses across sex, 9% across race/ethnicity/culture, and 4% across socioeconomic status. No subgroup analyses were reported for place of residence, occupation, religion, education, or social capital. One hundred and forty-one trials (71%) discussed the applicability of their results to one or more socially disadvantaged populations (68% of CRT, 73% of RCT). Discussion In this set of trials, selected for their relevance to health equity, data that were disaggregated for socially disadvantaged populations were rarely reported. We found that even when the data are available, opportunities to analyze health-equity considerations are frequently missed. The recently published equity extension of the Consolidated Reporting Standards for Randomized Trials (CONSORT-Equity) may help improve delineation of hypotheses related to socially disadvantaged populations, and transparency and completeness of reporting of health-equity considerations in RCTs. This study can serve as a baseline assessment of the reporting of equity considerations.

Abstract Background As systematic reviews’ limited coverage of the medical literature necessitates decision-making based on unsystematic review, we investigated a possible advantage of systematic review (aside from dataset size and systematic analysis): does systematic review avoid potential bias in sampling primary studies from high impact factor journals? If randomized controlled trials (RCTs) reported in higher-impact journals present different treatment benefits than RCTs reported in lower-impact journals, readers who focus on higher-impact journals for their rapid literature reviews may introduce bias which could be mitigated by complete, systematic sampling. Methods We randomly sampled Cochrane Library (20 July 2005) treatment reviews that measured mortality as a binary outcome, published in English or French, with at least five RCTs with one or more deaths. Our domain-based assessment of risk of bias included funding source, randomness of allocation sequence, blinding, and allocation concealment. The primary analysis employed logistic regression by a generalized linear model with a generalized estimating equation to estimate the association between various factors and publication in a journal with a high journal impact factor (JIF). Results From the 29 included systematic reviews, 189 RCTs contributed data. However, in the primary analyses comparing RCT results within meta-analyses, there was no statistically significant association: unadjusted odds of greater than 50% mortality protection in high-JIF (> 5) journals were 1.4 (95% CI 0.42, 4.4) and adjusted, 2.5 (95% CI 0.6, 10). Elements of study quality were weakly, inconsistently, and not statistically significantly correlated with journal impact factor. Conclusions Journal impact factor may have little to no association with study results, or methodological quality, but the evidence is very uncertain.

Abstract Background The lack of a reporting guideline for scaling of evidence-based practices (EBPs) studies has prompted the registration of the Standards for reporting studies assessing the impact of scaling strategies of EBPs (SUCCEED) with EQUATOR Network. The development of SUCCEED will be guided by the following main steps recommended for developing health research reporting guidelines. Methods Executive Committee. We established a committee composed of members of the core research team and of an advisory group. Systematic review. The protocol was registered with the Open Science Framework on 29 November 2019 (https://osf.io/vcwfx/). We will include reporting guidelines or other reports that may include items relevant to studies assessing the impact of scaling strategies. We will search the following electronic databases: EMBASE, PsycINFO, Cochrane Library, CINAHL, Web of Science, from inception. In addition, we will systematically search websites of EQUATOR and other relevant organizations. Experts in the field of reporting guidelines will also be contacted. Study selection and data extraction will be conducted independently by two reviewers. A narrative analysis will be conducted to compile a list of items for the Delphi exercise. Consensus process . We will invite panelists with expertise in: development of relevant reporting guidelines, methodologists, content experts, patient/member of the public, implementers, journal editors, and funders. We anticipated that three rounds of web-based Delphi consensus will be needed for an acceptable degree of agreement. We will use a 9-point scale (1 = extremely irrelevant to 9 = extremely relevant). Participants’ response will be categorized as irrelevant (1–3), equivocal (4–6) and relevant (7–9). For each item, the consensus is reached if at least 80% of the participants’ votes fall within the same category. The list of items from the final round will be discussed at face-to-face consensus meeting. Guideline validation . Participants will be authors of scaling studies. We will collect quantitative (questionnaire) and qualitative (semi-structured interview) data. Descriptive analyses will be conducted on quantitative data and constant comparative techniques on qualitative data. Discussion Essential items for reporting scaling studies will contribute to better reporting of scaling studies and facilitate the transparency and scaling of evidence-based health interventions.

Abstract Background Randomised controlled trials (RCTs) provide the most reliable information to inform clinical practice and patient care. We aimed to map global clinical research publication activity through RCT-related articles in high-impact-factor medical journals over the past five decades. Methods We conducted a cross-sectional analysis of articles published in the highest ranked medical journals with an impact factor > 10 (according to Journal Citation Reports published in 2017). We searched PubMed/MEDLINE (from inception to December 31, 2017) for all RCT-related articles (e.g. primary RCTs, secondary analyses and methodology papers) published in high-impact-factor medical journals. For each included article, raw metadata were abstracted from the Web of Science. A process of standardization was conducted to unify the different terms and grammatical variants and to remove typographical, transcription and/or indexing errors. Descriptive analyses were conducted (including the number of articles, citations, most prolific authors, countries, journals, funding sources and keywords). Network analyses of collaborations between countries and co-words are presented. Results We included 39,305 articles (for the period 1965–2017) published in forty journals. The Lancet (n = 3593; 9.1%), the Journal of Clinical Oncology (n = 3343; 8.5%) and The New England Journal of Medicine (n = 3275 articles; 8.3%) published the largest number of RCTs. A total of 154 countries were involved in the production of articles. The global productivity ranking was led by the United States (n = 18,393 articles), followed by the United Kingdom (n = 8028 articles), Canada (n = 4548 articles) and Germany (n = 4415 articles). Seventeen authors who had published 100 or more articles were identified; the most prolific authors were affiliated with Duke University (United States), Harvard University (United States) and McMaster University (Canada). The main funding institutions were the National Institutes of Health (United States), Hoffmann-La Roche (Switzerland), Pfizer (United States), Merck Sharp & Dohme (United States) and Novartis (Switzerland). The 100 most cited RCTs were published in nine journals, led by The New England Journal of Medicine (n = 78 articles), The Lancet (n = 9 articles) and JAMA (n = 7 articles). These landmark contributions focused on novel methodological approaches (e.g. the “Bland-Altman method”) and trials on the management of chronic conditions (e.g. diabetes control, hormone replacement therapy in postmenopausal women, multiple therapies for diverse cancers, cardiovascular therapies such as lipid-lowering statins, antihypertensive medications, and antiplatelet and antithrombotic therapy). Conclusions Our analysis identified authors, countries, funding institutions, landmark contributions and high-impact-factor medical journals publishing RCTs. Over the last 50 years, publication production in leading medical journals has increased, with Western countries leading in research but with low- and middle-income countries showing very limited representation.

Abstract Background Two reviews and an overview were produced for the Canadian Task Force on Preventive Health Care guideline on screening for esophageal adenocarcinoma in patients with chronic gastroesophageal reflux disease (GERD) without alarm symptoms. The goal was to systematically review three key questions (KQs): (1) The effectiveness of screening for these conditions; (2) How adults with chronic GERD weigh the benefits and harms of screening, and what factors contribute to their preferences and decision to undergo screening; and (3) Treatment options for Barrett’s esophagus (BE), dysplasia or stage 1 EAC (overview of reviews). Methods Bibliographic databases (e.g. Ovid MEDLINE®) were searched for each review in October 2018. We also searched for unpublished literature (e.g. relevant websites). The liberal accelerated approach was used for title and abstract screening. Two reviewers independently screened full-text articles. Data extraction and risk of bias assessments were completed by one reviewer and verified by another reviewer (KQ1 and 2). Quality assessments were completed by two reviewers independently in duplicate (KQ3). Disagreements were resolved through discussion. We used various risk of bias tools suitable for study design. The GRADE framework was used for rating the certainty of the evidence. Results Ten studies evaluated the effectiveness of screening. One retrospective study reported no difference in long-term survival (approximately 6 to 12 years) between those who had a prior esophagogastroduodenoscopy and those who had not (adjusted HR 0.93, 95% confidence interval (CI) 0.58–1.50). Though there may be higher odds of a stage 1 diagnosis than a more advanced diagnosis (stage 2–4) if an EGD had been performed in the previous 5 years (OR 2.27, 95% CI 1.00–7.67). Seven studies compared different screening modalities, and showed little difference between modalities. Three studies reported on patients’ unwillingness to be screened (e.g. due to anxiety, fear of gagging). Eleven systematic reviews evaluated treatment modalities, providing some evidence of early treatment effect for some outcomes. Conclusions Little evidence exists on the effectiveness of screening and values and preferences to screening. Many treatment modalities have been evaluated, but studies are small. Overall, there is uncertainty in understanding the effectiveness of screening and early treatments. Systematic review registrations PROSPERO (CRD42017049993 [KQ1], CRD42017050014 [KQ2], CRD42018084825 [KQ3]).

[This corrects the article DOI: 10.1371/journal.pbio.3001133.].

Alongside the growing concerns regarding predatory journal growth, other questionable editorial practices have gained visibility recently. Among them, we explored the usefulness of the Percentage of Papers by the Most Prolific author (PPMP) and the Gini index (level of inequality in the distribution of authorship among authors) as tools to identify journals that may show favoritism in accepting articles by specific authors. We examined whether the PPMP, complemented by the Gini index, could be useful for identifying cases of potential editorial bias, using all articles in a sample of 5,468 biomedical journals indexed in the National Library of Medicine. For articles published between 2015 and 2019, the median PPMP was 2.9%, and 5% of journal exhibited a PPMP of 10.6% or more. Among the journals with the highest PPMP or Gini index values, where a few authors were responsible for a disproportionate number of publications, a random sample was manually examined, revealing that the most prolific author was part of the editorial board in 60 cases (61%). The papers by the most prolific authors were more likely to be accepted for publication within 3 weeks of their submission. Results of analysis on a subset of articles, excluding nonresearch articles, were consistent with those of the principal analysis. In most journals, publications are distributed across a large number of authors. Our results reveal a subset of journals where a few authors, often members of the editorial board, were responsible for a disproportionate number of publications. To enhance trust in their practices, journals need to be transparent about their editorial and peer review practices.

Florian Naudet and co-authors discuss strengthening requirements for sharing clinical trial data.

Objectives To explore the impact of data-sharing initiatives on the intent to share data, on actual data sharing, on the use of shared data and on research output and impact of shared data.Eligibility criteria All studies investigating data-sharing practices for individual participant data (IPD) from clinical trials.Sources of evidence We searched the Medline database, the Cochrane Library, the Science Citation Index Expanded and the Social Sciences Citation Index via Web of Science, and preprints and proceedings of the International Congress on Peer Review and Scientific Publication. In addition, we inspected major clinical trial data-sharing platforms, contacted major journals/publishers, editorial groups and some funders.Charting methods Two reviewers independently extracted information on methods and results from resources identified using a standardised questionnaire. A map of the extracted data was constructed and accompanied by a narrative summary for each outcome domain.Results 93 studies identified in the literature search (published between 2001 and 2020, median: 2018) and 5 from additional information sources were included in the scoping review. Most studies were descriptive and focused on early phases of the data-sharing process. While the willingness to share IPD from clinical trials is extremely high, actual data-sharing rates are suboptimal. A survey of journal data suggests poor to moderate enforcement of the policies by publishers. Metrics provided by platforms suggest that a large majority of data remains unrequested. When requested, the purpose of the reuse is more often secondary analyses and meta-analyses, rarely re-analyses. Finally, studies focused on the real impact of data-sharing were rare and used surrogates such as citation metrics.Conclusions There is currently a gap in the evidence base for the impact of IPD sharing, which entails uncertainties in the implementation of current data-sharing policies. High level evidence is needed to assess whether the value of medical research increases with data-sharing practices.

Background: Protocols are an essential document for conducting randomised controlled trials (RCTs). However, the completeness of the information provided is often inadequate. To help improve the content of trial protocols, an international group of stakeholders published the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Initiative in 2013. Presently, there is increasing use of cohorts and routinely collected data (RCD) for RCTs because these data have the potential to improve efficiencies by facilitating recruitment, simplifying, and reducing the cost of data collection. Reporting guidelines have been shown to improve the quality of reporting, but there is currently no specific SPIRIT guidance on protocols for trials conducted using cohorts and RCD. This protocol outlines steps for developing SPIRIT-ROUTINE, which aims to address this gap by extending the SPIRIT guidance to protocols for trials conducted using cohorts and RCD. Methods: The development of the SPIRIT-ROUTINE extension comprises five stages. Stage 1 consists of a project launch and a meeting to finalise the membership of the steering group and scope of the extension. In Stage 2, a rapid review will be performed to identify possible modifications to the original SPIRIT 2013 checklist. Other key reporting guidelines will be reviewed to identify areas where additional items may be needed, such as the Consolidated Standards of Reporting Trials (CONSORT) extension for trials conducted using cohorts and RCD (CONSORT-ROUTINE). Stage 3 will involve an online Delphi exercise, consisting of two rounds and involving key international stakeholders to gather feedback on the preliminary checklist items. In Stage 4, a consensus meeting of the SPIRIT-ROUTINE steering group will finalise the items to include in the extension. Stage 5 will involve the publication preparation and dissemination of the final checklist. Conclusion: The SPIRIT-ROUTINE extension will contribute to improving design of trials using cohorts and RCD and transparency of reporting.

Abstract Background The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process. Methods We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9). Results The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal’s usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88–1.84] (p

Abstract Background Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals. Methods Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes. Results After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers’ tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items). Conclusions Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.

Resumen: Objetivo: Examinar el apoyo a las guías de publicación para estudios de evaluación económica, como la declaración CHEERS (Consolidated Health Economic Evaluation Reporting Standards), por parte de las revistas biomédicas españolas. Método: Análisis transversal de las normas de autoría de las revistas biomédicas españolas incluidas en Journal Citation Reports 2017. Dos autores examinaron y extrajeron la siguiente información: mención de alguna guía de publicación, declaración CHEERS, recomendaciones del International Committee of Medical Journal Editors (ICMJE) e iniciativa Enhancing the QUAlity and Transparency Of health Research (EQUATOR). Resultados: De las 28 revistas incluidas, 23 (82,1%; intervalo de confianza del 95% [IC95%]: 63,1-93,9%) mencionaron alguna guía. Una única revista mencionó la declaración CHEERS para estudios de evaluación económica. Veinticuatro revistas (85,7%; IC95%: 67,3-96,0%) mencionaron las recomendaciones del ICMJE y 8 (28,6%; IC95%: 13,2-48,7%) la red EQUATOR. La declaración CONSORT (Consolidated Standards of Reporting Trials) para ensayos clínicos fue la guía más mencionada (n = 21; 75,0%; IC95%: 55,1-89,3%). Discusión: La mayoría de las normas de autoría no incorporan información sobre cómo presentar evaluaciones económicas. Las revistas deberían apoyar el cumplimiento de las guías de publicación por parte de las personas autoras y revisoras. Abstract: Objective: To examine the endorsement of reporting guidelines for economic evaluation studies, such as the CHEERS (Consolidated Health Economic Evaluation Reporting Standards) statement, by Spanish biomedical journals. Method: Cross-sectional analysis of the instructions to authors of Spanish biomedical journals included in the Journal Citation Reports 2017. Two authors examined and extracted the following information: mention of any reporting guideline, the CHEERS statement, the recommendations of the International Committee of Medical Journal Directors (ICMJE) and the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. Results: Of the 28 journals included, 23 (82.1%; 95% confidence interval [95%CI]: 63.1-93.9%) mentioned at least one reporting guideline in the instructions to authors. Only one journal mentioned the CHEERS statement for health economic evaluations. Twenty-four journals (85.7%; 95%CI: 67.3-96.0%) mentioned the ICMJE recommendations and 8 (28.6%; 95%CI: 13.2-48.7%) mentioned the EQUATOR network. The CONSORT (Consolidated Standards of Reporting Trials) statement for clinical trials was the most- mentioned reporting guideline (n = 21; 75.0%; 95%CI: 55.1-89.3%). Discussion: Most of the instructions to authors do not provide guidance on how to report economic evaluations. Journals should support compliance with reporting guidelines by authors and peer-reviewers. Palabras clave: CHEERS, Coste-efectividad, Guía de publicación, Keywords: CHEERS, Cost-effectiveness, Reporting guideline

Abstract Background Inadequate and poor quality outcome reporting in clinical trials is a well-documented problem that impedes the ability of researchers to evaluate, replicate, synthesize, and build upon study findings and impacts evidence-based decision-making by patients, clinicians, and policy-makers. To facilitate harmonized and transparent reporting of outcomes in trial protocols and published reports, the Instrument for reporting Planned Endpoints in Clinical Trials (InsPECT) is being developed. The final product will provide unique InsPECT extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines. Methods The InsPECT SPIRIT and CONSORT extensions will be developed in accordance with the methodological framework created by the EQUATOR (Enhancing the Quality and Transparency of Health Research Quality) Network for reporting guideline development. Development will consist of (1) the creation of an initial list of candidate outcome reporting items synthesized from expert consultations and a scoping review of existing guidance for reporting outcomes in trial protocols and reports; (2) a three-round international Delphi study to identify additional candidate items and assess candidate item importance on a 9-point Likert scale, completed by stakeholders such as trial report and protocol authors, systematic review authors, biostatisticians and epidemiologists, reporting guideline developers, clinicians, journal editors, and research ethics board representatives; and (3) an in-person expert consensus meeting to finalize the set of essential outcome reporting items for trial protocols and reports, respectively. The consensus meeting discussions will be independently facilitated and informed by the empirical evidence identified in the primary literature and through the opinions (aggregate rankings and comments) collected via the Delphi study. An integrated knowledge translation approach will be used throughout InsPECT development to facilitate implementation and dissemination, in addition to standard post-development activities. Discussion InsPECT will provide evidence-informed and consensus-based standards focused on outcome reporting in clinical trials that can be applied across diverse disease areas, study populations, and outcomes. InsPECT will support the standardization of trial outcome reporting, which will maximize trial usability, reduce bias, foster trial replication, improve trial design and execution, and ultimately reduce research waste and help improve patient outcomes.

Abstract Background In 2018, the World Health Organization reported that depression is the most common cause of disability worldwide, with over 300 million people currently living with depression. Depression affects an individual’s physical health and well-being, impacts psychosocial functioning, and has specific negative short- and long-term effects on maternal health, child health, developmental trajectories, and family health. The aim of these reviews is to identify evidence on the benefits and harms of screening for depression in the general adult population and in pregnant and postpartum women. Methods Search strategies were developed and tested through an iterative process by an experienced medical information specialist in consultation with the review team. We will search MEDLINE, Embase, PsycINFO, CINAHL, and the Cochrane Library, and a randomized controlled trial filter will be used. The general adult review will be an update of a systematic review previously used by the Canadian Task Force on Preventive Health Care for their 2013 guideline recommendation. The search strategy will be updated and will start from the last search date of the previous review (May 2012). The pregnant and postpartum review will be a de novo review with no date restriction. For both reviews, we will search for unpublished documents following the CADTH Grey Matters checklist and relevant websites. Titles and abstracts will be screened using the liberal accelerated method. Two reviewers will independently screen full-text articles for relevance using pre-specified eligibility criteria and assess the risk of bias of included studies using the Cochrane Risk of Bias tool. Outcomes of interest for the general adult population review include symptoms of depression or diagnosis of major depressive disorder, health-related quality of life, day-to-day functionality, lost time at work/school, impact on lifestyle behaviour, suicidality, false-positive result, labelling/stigma, overdiagnosis or overtreatment, and harms of treatment. Outcomes of interest for the pregnant and postpartum review include mental health outcomes (e.g. diagnosis of major depressive disorder), parenting outcomes (e.g. mother-child interactions), and infant outcomes (e.g. infant health and development). Discussion These two systematic reviews will offer informative evaluations of depression screening. The findings will be used by the Task Force to help develop guideline recommendations on depression screening in the general adult population and in pregnant and postpartum women in Canada. Systematic review registration PROSPERO (CRD42018099689, CRD42018099690)

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.

Background: Funded health research is being published in journals that many regard as “predatory”, deceptive, and non-credible. We do not currently know whether funders provide guidance on how to select a journal in which to publish funded health research. Methods: We identified the largest 46 philanthropic, public, development assistance, public-private partnership, and multilateral funders of health research by expenditure, globally as well as four public funders from lower-middle income countries, from the list at https://healthresearchfunders.org. One of us identified guidance on disseminating funded research from each funders’ website (August/September 2017), then extracted information about selecting journals, which was verified by another assessor. Discrepancies were resolved by discussion. Results were summarized descriptively. This research used publicly available information; we did not seek verification with funding bodies. Results: The majority (44/50) of sampled funders indicated funding health research. 38 (of 44, 86%) had publicly available information about disseminating funded research, typically called “policies” (29, 76%). Of these 38, 36 (95%) mentioned journal publication for dissemination of which 13 (36.11%) offer variable guidance on selecting a journal, all of which relate to the funder’s open access mandate. Six funders (17%) outlined publisher requirements or features by which to select a journal. One funder linked to a document providing features of journals to look for (e.g. listed in the Directory of Open Access Journals) and to be wary of (e.g., no journal scope statement, uses direct and unsolicited marketing). Conclusions: Few funders provided guidance on how to select a journal in which to publish funded research. Funders have a duty to ensure that the research they fund is discoverable by others. This research is a benchmark for funder guidance on journal selection prior to the January 2021 implementation of Plan S (a global, funder-led initiative to ensure immediate, open access to funded, published research).

In this response to Labib and Evans, authors of the Hong Kong Principles look forward to collaborating with those from the broad research integrity community to ensure that issues of equity, diversity and inclusion will become part of the ecosystem of research integrity.

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.

Matthew Page and co-authors describe PRISMA 2020, an updated reporting guideline for systematic reviews and meta-analyses.

Introduction Methodological studies (ie, studies that evaluate the design, conduct, analysis or reporting of other studies in health research) address various facets of health research including, for instance, data collection techniques, differences in approaches to analyses, reporting quality, adherence to guidelines or publication bias. As a result, methodological studies can help to identify knowledge gaps in the methodology of health research and strategies for improvement in research practices. Differences in methodological study names and a lack of reporting guidance contribute to lack of comparability across studies and difficulties in identifying relevant previous methodological studies. This paper outlines the methods we will use to develop an evidence-based tool—the MethodologIcal STudy reportIng Checklist—to harmonise naming conventions and improve the reporting of methodological studies.Methods and analysis We will search for methodological studies in the Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Embase, MEDLINE, Web of Science, check reference lists and contact experts in the field. We will extract and summarise data on the study names, design and reporting features of the included methodological studies. Consensus on study terms and recommended reporting items will be achieved via video conference meetings with a panel of experts including researchers who have published methodological studies.Ethics and dissemination The consensus study has been exempt from ethics review by the Hamilton Integrated Research Ethics Board. The results of the review and the reporting guideline will be disseminated in stakeholder meetings, conferences, peer-reviewed publications, in requests to journal editors (to endorse or make the guideline a requirement for authors), and on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network and reporting guideline websites.Registration We have registered the development of the reporting guideline with the EQUATOR Network and publicly posted this project on the Open Science Framework (www.osf.io/9hgbq).

Objectives The objective of this review is to identify all preprint platforms with biomedical and medical scope and to compare and contrast the key characteristics and policies of these platforms.Study design and setting Preprint platforms that were launched up to 25 June 2019 and have a biomedical and medical scope according to MEDLINE’s journal selection criteria were identified using existing lists, web-based searches and the expertise of both academic and non-academic publication scientists. A data extraction form was developed, pilot tested and used to collect data from each preprint platform’s webpage(s).Results A total of 44 preprint platforms were identified as having biomedical and medical scope, 17 (39%) were hosted by the Open Science Framework preprint infrastructure, 6 (14%) were provided by F1000 Research (the Open Research Central infrastructure) and 21 (48%) were other independent preprint platforms. Preprint platforms were either owned by non-profit academic groups, scientific societies or funding organisations (n=28; 64%), owned/partly owned by for-profit publishers or companies (n=14; 32%) or owned by individuals/small communities (n=2; 5%). Twenty-four (55%) preprint platforms accepted content from all scientific fields although some of these had restrictions relating to funding source, geographical region or an affiliated journal’s remit. Thirty-three (75%) preprint platforms provided details about article screening (basic checks) and 14 (32%) of these actively involved researchers with context expertise in the screening process. Almost all preprint platforms allow submission to any peer-reviewed journal following publication, have a preservation plan for read access and most have a policy regarding reasons for retraction and the sustainability of the service.Conclusion A large number of preprint platforms exist for use in biomedical and medical sciences, all of which offer researchers an opportunity to rapidly disseminate their research findings onto an open-access public server, subject to scope and eligibility.

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited

Summary: The CONSORT 2010 statement provides minimum guidelines for reporting randomised trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders), and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human–AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret, and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

Background: The process of translating preclinical findings into a clinical setting takes decades. Previous studies have suggested that only 5-10% of the most promising preclinical studies are successfully translated into viable clinical applications. The underlying determinants of this low success rate (e.g. poor experimental design, suboptimal animal models, poor reporting) have not been examined in an empirical manner. Our study aims to determine the contemporary success rate of preclinical-to-clinical translation, and subsequently determine if an association between preclinical study design and translational success/failure exists. Methods: Established systematic review methodology will be used with regards to the literature search, article screening and study selection process. Preclinical, basic science studies published in high impact basic science journals between 1995 and 2015 will be included. Included studies will focus on publicly available interventions with potential clinical promise. The primary outcome will be successful clinical translation of promising therapies - defined as the conduct of at least one Phase II trial (or greater) with a positive finding. A case-control study will then be performed to evaluate the association between elements of preclinical study design and reporting and the likelihood of successful translation. Discussion: This study will provide a comprehensive analysis of the therapeutic translation from the laboratory bench to the bedside. Importantly, any association between factors of study design and the success of translation will be identified. These findings may inform future research teams attempting preclinical-to-clinical translation. Results will be disseminated to identified knowledge users that fund/support preclinical research.