Search

Objectives To assess the impact of restricting systematic reviews of conventional or alternative medical treatments or diagnostic tests to English-language publications. Study design and setting We systematically searched MEDLINE (Ovid), the Science Citation Index Expanded (Web of Science), and Current Contents Connect (Web of Science) up to April 24, 2020. Eligible methods studies assessed the impact of restricting systematic reviews to English-language publications on effect estimates and conclusions. Two reviewers independently screened the literature; one investigator performed the data extraction, a second investigator checked for completeness and accuracy. We synthesized the findings narratively. Results Eight methods studies (10 publications) met the inclusion criteria; none addressed language restrictions in diagnostic test accuracy reviews. The included studies analyzed nine to 147 meta-analyses and/or systematic reviews. The proportions of non-English-language publications ranged from 2% to 100%. Based on five methods studies, restricting literature searches or inclusion criteria to English-language publications led to a change in statistical significance in 23/259 meta-analyses (9%). Most commonly, the statistical significance was lost, but had no impact on the conclusions of systematic reviews. Conclusion Restricting systematic reviews to English-language publications appears to have little impact on the effect estimates and conclusions of systematic reviews.

Objectives: We aimed to assess whether limiting the inclusion criteria solely to English-language publications affected the overall conclusions of evidence syntheses.Study Design and Setting: Our analyses used a dataset of a previous methods study that included 59 randomly selected Cochrane intervention reviews with no language restrictions. First, we ascertained the publication language of all 2,026 included publications. Next, we excluded studies based on the following criteria: (1) publication solely in non-English language, or (2) main publication (in case of multiple publications of the same study) in non-English language. We then re-calculated meta-analyses for outcomes that were presented in the main summary of findings tables of the Cochrane reports. If the direction of the effect estimate or the statistical significance changed, authors of the respective Cochrane reviews were consulted to assess whether the new evidence base would have changed their conclusions. The primary outcome of our analyses examined the proportion of conclusions that would change with the exclusion of non-English publications. We set the threshold for the approach as noninferior if the upper limit of the 95% confidence interval of the proportion of changed conclusions did not cross a margin of 10%.Results: Across all 59 Cochrane reviews, 29 (49%) included 80 non-English publications. For 16 (27%) of these Cochrane reviews, the exclusion of non-English publications resulted in the exclusion of at least one study. In the remaining 13 Cochrane reviews, the non-English publications were not the only or main publication of the study or they did not contribute to the main summary of the findings table, so their exclusion did not result in an exclusion of the study.Overall, the exclusion of non-English publications led to the exclusion of 31 studies contributing to 40 outcomes. For 38 of the 40 outcomes, the exclusion of non-English studies did not markedly alter the size or direction of effect estimates or statistical significance. In two outcomes, the statistical significance changed, but authors would have still drawn the same conclusion, albeit with less certainty. Thus, the proportion of changed conclusions in our sample was 0.0% (95% CI 0.0-0.6), which indicated the noninferiority of the approach. However, the majority of excluded studies were small.Conclusion: Exclusion of non-English publications from systematic reviews on clinical interventions had a minimal effect on overall conclusions and could be a viable methodological shortcut, especially for rapid reviews. (C) 2019 Elsevier Inc. All rights reserved.

BACKGROUND AND OBJECTIVES: Assessing changes in coverage, recall, review, conclusions and references not found when searching fewer databases. METHODS: In randomly selected 60 Cochrane reviews, we checked included study publications' coverage (indexation) and recall (findability) using different search approaches with MEDLINE, Embase, and CENTRAL and related them to authors' conclusions and certainty. We assessed characteristics of unfound references. RESULTS: Overall 1989/2080 included references, were indexed in ≥1 database (coverage = 96%). In reviews where using one of our search approaches would not change conclusions and certainty (n = 44-54), median coverage and recall were highest (range 87.9%-100.0% and 78.2%-93.3%, respectively). Here, searching ≥2 databases reached >95% coverage and ≥87.9% recall. In reviews with unchanged conclusions but less certainty (n = 2-8): 63.3%-79.3% coverage and 45.0%-75.0% recall. In reviews with opposite conclusions (n = 1-3): 63.3%-96.6% and 52.1%-78.7%. In reviews where a conclusion was no longer possible (n = 3-7): 60.6%-86.0% and 20.0%-53.8%. The 265 references that were indexed but unfound were more often abstractless (30% vs. 11%) and older (28% vs. 17% published before 1991) than found references. CONCLUSION: Searching ≥2 databases improves coverage and recall and decreases the risk of missing eligible studies. If researchers suspect that relevant articles are difficult to find, supplementary search methods should be used.

Background and Objectives Rapid reviews provide an efficient alternative to standard systematic reviews in response to a high priority or urgent need. Although rapid reviews of interventions have been extensively evaluated, little is known about the characteristics of rapid reviews of diagnostic evidence. Study Design and Setting We performed a scoping review for rapid reviews of medical tests published from 2013 to 2018. We extracted information on review characteristics and methods used to assess the evidence. Results We identified 191 rapid reviews. All reviews were developed within a short time (less than 12 months) and were relatively concise (less than 10 pages). The reviews involved multiple index tests (44%), multiple outcomes (88%), and several test applications (29%). Well-known methodological tailoring strategies were infrequently used. Although reporting of several key features was limited, we found that, in general, rapid reviews have similar characteristics to broader knowledge syntheses. Conclusion Our scoping review is the first to describe the characteristics and methods of rapid reviews of diagnostic evidence. Future research should identify the most appropriate methods for performing rapid reviews of medical tests. Standards for reporting of rapid reviews are needed.

Objectives The objective is to identify studies that have assessed methodological shortcuts for undertaking rapid reviews (RRs) and mapping these to review conduct stages and Methodological Expectations of Cochrane Intervention Reviews (MECIR) guidance. Study Design and Setting We conducted a systematic scoping review. We searched multiple databases (e.g., MEDLINE, Embase), which were supplemented by grey literature searching. Methods were defined a priori in a published protocol. Results Out of 1,873 records, 90 publications were divided into four RR categories: formal evaluation (n = 14), development, which included four subcategories (n = 65), comparison (n = 2), and applying reporting guidelines/critical appraisal tools (n = 3), and a systematic review surrogate category (n = 6). Four formal evaluation studies were composite evaluations, including more than one shortcut simultaneously. The remaining 10 studies evaluated viable (e.g., including English-only publications) and unviable (e.g., single-reviewer screening) shortcuts, covering five key dimensions and five ‘other’ (e.g., involving stakeholders) considerations while conducting a review. Because of complexities around shortcuts evaluated, only a cursory mapping to MECIR criteria was possible. Conclusion Some methods shortcuts may be valid in the context of RRs, but limitations in the studies may limit their applicability. The results will serve to inform discussions within Cochrane regarding possible future implementation of RRs.

Objective To assess the effects of abbreviated literature searches on evidence syntheses conclusions. Study design and setting We randomly selected 60 Cochrane reviews of clinical interventions and repeated literature searches using 14 abbreviated approaches (combinations of MEDLINE, Embase, CENTRAL with and without searches of reference lists). If abbreviated searches missed included studies, we recalculated meta-analyses. Cochrane authors determined whether the new evidence base would change conclusions. We assessed the noninferiority of abbreviated searches allowing for a maximum of 10% changed conclusions. Results We conducted 840 abbreviated literature searches. Noninferiority varied based on the definition of “changed conclusion”. When the reduction of the certainty of a conclusion was of concern, all abbreviated searches were inferior. Searching Embase only rendered the greatest proportion of changed conclusions (27%, 95% confidence interval [CI]: 16%–40%); combining MEDLINE, Embase, CENTRAL with searches of references lists the lowest (8%, 95% CI 3%–18%). When falsely reaching an opposite conclusion was of concern, combining one database with another or with searches of reference lists was noninferior to comprehensive searches (2%, 95% CI: 0%–9%). Conclusion If decision-makers are willing to accept less certainty and a small risk for opposite conclusions, some abbreviated searches are viable options for rapid evidence syntheses. Decisions demanding high certainty require comprehensive searches.

Objectives The aim of article was to assess the risk for random errors in outcomes graded as high certainty of evidence (CoE). Study Design and Setting We randomly selected 100 Cochrane reviews with dichotomous outcomes rated as high CoE using Grading of Recommendations Assessment, Development, and Evaluation. To detect increased risks for random errors, two investigators independently conducted trial sequential analysis using conventional thresholds for type I (α = 0.05) and type II (β = 0.10) errors. We dually regraded all outcomes with increased risks for random errors and conducted multivariate logistic regression analyses to determine predictors of increased risks for random errors. Results Overall, 38% (95% confidence interval: 28–47%) of high CoE outcomes had increased risks for random errors. Outcomes assessing harms were more frequently affected than outcomes assessing benefits (47% vs. 12%). Regrading of outcomes with increased random errors showed that 74% should have been downgraded based on current guidance. Regression analyses rendered small absolute risk differences (P = 0.009) and low number of events (P = 0.001) as significant predictors of increased risks for random errors. Conclusion Decisionmakers need to be aware that outcomes rated as high CoE often have increased risks for false-positive or false-negative findings.