Search

Abstract Background Systematic reviews (SRs) are invaluable evidence syntheses, widely used in biomedicine and other scientific areas. Tremendous resources are being spent on the production and updating of SRs. There is a continuous need to automatize the process and use the workforce and resources to make it faster and more efficient. Methods Information gathered by previous EVBRES research was used to construct a questionnaire for round 1 which was partly quantitative, partly qualitative. Fifty five experienced SR authors were invited to participate in a Delphi study (DS) designed to identify the most promising areas and methods to improve the efficient production and updating of SRs. Topic questions focused on which areas of SRs are most time/effort/resource intensive and should be prioritized in further research. Data were analysed using NVivo 12 plus, Microsoft Excel 2013 and SPSS. Thematic analysis findings were used on the topics on which agreement was not reached in round 1 in order to prepare the questionnaire for round 2. Results Sixty percent (33/55) of the invited participants completed round 1; 44% (24/55) completed round 2. Participants reported average of 13.3 years of experience in conducting SRs (SD 6.8). More than two thirds of the respondents agreed/strongly agreed the following topics should be prioritized: extracting data, literature searching, screening abstracts, obtaining and screening full texts, updating SRs, finding previous SRs, translating non-English studies, synthesizing data, project management, writing the protocol, constructing the search strategy and critically appraising. Participants have not considered following areas as priority: snowballing, GRADE-ing, writing SR, deduplication, formulating SR question, performing meta-analysis. Conclusions Data extraction was prioritized by the majority of participants as an area that needs more research/methods development. Quality of available language translating tools has dramatically increased over the years (Google translate, DeepL). The promising new tool for snowballing emerged (Citation Chaser). Automation cannot substitute human judgement where complex decisions are needed (GRADE-ing). Trial registration Study protocol was registered at https://osf.io/bp2hu/ .

Abstract Background Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers. Methods This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review. Results The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication. Conclusions Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

Abstract Background Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice. Methods We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere. Results Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read. Conclusions Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored. Protocol availability The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

Abstract Background Decision-makers increasingly request rapid answers to clinical or public health questions. To save time, personnel, and financial resources, rapid reviews streamline the methodological steps of the systematic review process. We aimed to explore the validity of a rapid review approach that combines a substantially abbreviated literature search with a single-reviewer screening of abstracts and full texts using three case studies. Methods We used a convenience sample of three ongoing Cochrane reviews as reference standards. Two reviews addressed oncological topics and one addressed a public health topic. For each of the three topics, three reviewers screened the literature independently. Our primary outcome was the change in conclusions between the rapid reviews and the respective Cochrane reviews. In case the rapid approach missed studies, we recalculated the meta-analyses for the main outcomes and asked Cochrane review authors if the new body of evidence would change their original conclusion compared with the reference standards. Additionally, we assessed the sensitivity of the rapid review approach compared with the results of the original Cochrane reviews. Results For the two oncological topics (case studies 1 and 2), the three rapid reviews each yielded the same conclusions as the Cochrane reviews. However, the authors would have had less certainty about their conclusion in case study 2. For case study 3, the public health topic, only one of the three rapid reviews led to the same conclusion as the Cochrane review. The other two rapid reviews provided insufficient information for the authors to draw conclusions. Using the rapid review approach, the sensitivity was 100% (3 of 3) for case study 1. For case study 2, the three rapid reviews identified 40% (4 of 10), 50% (5 of 10), and 60% (6 of 10) of the included studies, respectively; for case study 3, the respective numbers were 38% (8 of 21), 43% (9 of 21), and 48% (10 of 21). Conclusions Within the limitations of these case studies, a rapid review approach that combines abbreviated literature searches with single-reviewer screening may be feasible for focused clinical questions. For complex public health topics, sensitivity seems to be insufficient.

Abstract Background Rapid reviews (RRs) have emerged as an efficient alternative to time-consuming systematic reviews—they can help meet the demand for accelerated evidence synthesis to inform decision-making in healthcare. The synthesis of diagnostic evidence has important methodological challenges. Here, we performed an international survey to identify the current practice of producing RRs for diagnostic tests. Methods We developed and administered an online survey inviting institutions that perform RRs of diagnostic tests from all over the world. Results All participants (N = 25) reported the implementation of one or more methods to define the scope of the RR; however, only one strategy (defining a structured question) was used by ≥90% of participants. All participants used at least one methodological shortcut including the use of a previous review as a starting point (92%) and the use of limits on the search (96%). Parallelization and automation of review tasks were not extensively used (48 and 20%, respectively). Conclusion Our survey indicates a greater use of shortcuts and limits for conducting diagnostic test RRs versus the results of a recent scoping review analyzing published RRs. Several shortcuts are used without knowing how their implementation affects the results of the evidence synthesis in the setting of diagnostic test reviews. Thus, a structured evaluation of the challenges and implications of the adoption of these RR methods is warranted.

Abstract Background Assessment of diagnostic tests, broadly defined as any element that aids in the collection of additional information for further clarification of a patient’s health status, has increasingly become a critical issue in health policy and decision-making. Diagnostic evidence, including the accuracy of a medical test for a target condition, is commonly appraised using standard systematic review methodology. Owing to the considerable time and resources required to conduct these, rapid reviews have emerged as a pragmatic alternative by tailoring methods according to the decision maker’s circumstances. However, it is not known if streamlining methodological aspects has an impact on the validity of evidence synthesis. Furthermore, due to the particular nature and complexity of the appraisal of diagnostic accuracy, there is need for detailed guidance on how to conduct rapid reviews of diagnostic tests. In this study, we aim to identify the methods currently used by rapid review developers to synthesize evidence on diagnostic test accuracy, as well as to analyze potential shortcomings and challenges related to these methods. Methods We will carry out a two-fold approach: (1) an international survey of professionals working in organizations that develop rapid reviews of diagnostic tests, in terms of the methods and resources used by these agencies when conducting rapid reviews, and (2) semi-structured interviews with senior-level individuals to further explore and validate the findings from the survey and to identify challenges in conducting rapid reviews. We will use STATA 15.0 for quantitative analyses and framework analysis for qualitative analyses. We will ensure protection of data during all stages. Discussion The main result of this research will be a map of methods and resources currently used for conducting rapid reviews of diagnostic test accuracy, as well as methodological shortcomings and potential solutions in diagnostic knowledge synthesis that require further research.

Abstract Background Periodontal diseases are responsible for a vast burden of disease globally and are associated with other severe illnesses such as cardiovascular diseases or diabetes. Tests for early diagnosis of periodontal diseases and effective treatments are available. The effectiveness of screening for periodontal diseases to detect periodontal diseases at an early stage during periodic health examinations at primary care facilities, however, is unclear. The objective of this systematic review is to assess the benefits and risks of screening for periodontal diseases in adults during the periodic health examinations. Methods We will use two methodological approaches: (1) a systematic review to assess the effectiveness and risk of harms of screening for periodontal diseases during periodic health examinations and (2) an overview of systematic reviews to determine the effectiveness of treatment approaches for early periodontal disease. We will search electronic databases (Ovid MEDLINE, Embase.com, the Cochrane Library, Epistemonikos, Centre for Reviews and Dissemination databases, PubMed (non-MEDLINE content)) for published studies as well as sources for grey literature to detect unpublished studies. Two authors will independently screen abstracts and full texts using pre-defined eligibility criteria, select studies, extract data, and assess the risk of bias of included studies or reviews. In general, we will conduct a systematic narrative synthesis. Criteria for conducting meta-analyses were defined a priori. Our primary outcomes of interest are tooth loss, loosening of teeth, and depletion of bone tissue. Secondary outcomes are gingivitis/gum inflammation, pocket depths, dental hygiene, lifestyle modifications (e.g., smoking, alcohol, nutrition), and toothache. We consulted a panel of experts and patient representatives to prioritize these outcomes. Two investigators will assess independently the certainty of the evidence for each outcome using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. Discussion We anticipate that our review will highlight the gaps in the available evidence about the effectiveness of screening for periodontal diseases during periodic health examinations. Implications for screening programs may be based on linked evidence about the validity of available screening tools and the effectiveness of early treatment. Systematic review registration PROSPERO CRD42017081150

Abstract Background Health promotion programs can only lead to improvements in health outcomes if they are effectively implemented. However, most studies assessing implementation success focus on only one condition, although more conditions influence this process. Therefore, evidence is scarce on what conditions play a role in successful implementation and how they interact. Hence, we aimed to identify which combinations of teacher and implementation process characteristics affected the emotional and social school experience (SCE) of pupils participating in a school-based health promotion program. Methods This study was part of an effectiveness and process evaluation including 24 intervention and 27 control classes. We used fuzzy-set qualitative comparative analysis (fsQCA) to identify combinations of conditions that were associated with either an increase or no increase in the outcome SCE in comparison to the control group at 20 months post intervention. We deductively selected five conditions based on the Consolidated Framework for Implementation Research: teachers’ perceived self-efficacy, teachers’ expectations of the benefits of the intervention, teachers’ previous knowledge about the intervention, dosage of physical activity breaks, and quality of the implementation. Results We identified five different pathways that led to no increase in the pupils’ outcome (parameters of fit: consistency 94%, coverage 66%). The combination of an unsatisfying quality of implementing the intervention and a low previous knowledge about the intervention showed the highest empirical relevance. Similarly, fewer physical activity breaks in combination with other conditions impeded the program’s success. Furthermore, we identified two different pathways characterizing ways to success (consistency: 81%, coverage: 52%). The most relevant combination was good quality implementation of physical activity breaks, implemented by teachers with a high self-efficacy, and a good previous knowledge about the intervention. Conclusions QCA has potential for an in-depth analysis of complex interventions as it can rely on small to medium sample sizes and analyze pathways to success and non-success separately. The investigated program can be improved by considering the following suggestions: The quality of the implementation process should be monitored during the implementation phase, and regular feedback loops and learning opportunities for teachers should accompany a program. Clear recommendations regarding the dosage should be established. Trial registration German register of clinical studies: DRKS00000622. Retrospectively registered: December 3, 2010, (http://www.drks.de/drks_web/setLocale_EN.do). Approved by the Ethics Committee of Lower Austria (GS4-EK-4/107–2010).

Abstract Background Seasonal affective disorder (SAD) is a seasonally recurrent type of major depression that has detrimental effects on patients’ lives during winter. Little is known about how it affects patients during summer and about patients’ and physicians’ perspectives on preventive SAD treatment. The aim of our study was to explore how SAD patients experience summers, what type of preventive treatment patients implement, which preventive treatment methods, if any, physicians recommend, and what factors facilitate or hinder implementation/recommendation of SAD prevention. Methods We conducted 15 semi-structured interviews, ten with adult patients with a history of SAD and five with physicians. Transcripts were analyzed by two researchers using an inductive thematic analysis approach. Results One group of patients was able to enjoy summer and ignore thoughts of the upcoming winter. The other group feared the impending depressive episode in winter, and this fear negatively impacted these patients’ well-being during the summer. Preventive treatment was a relevant issue for all patients, and all but one person implemented SAD prevention during summer. We identified six factors that influenced patient use of preventive treatment of SAD. Four factors occur on an individual level (knowledge about disease and preventive treatment options, experience with treatment in acute phase, acceptability of intervention, willingness to take responsibility for oneself), one on an interpersonal level (social and work environment), and one on a structural level (healthcare system). All psychiatrists recommended some kind of preventive intervention, most commonly, lifestyle changes. Four factors influenced psychiatrists in recommending prevention of SAD (patient expectations, disease history and stability, risk/benefit ratio, lack of evidence). Conclusions Success in the implementation of SAD prevention does not solely depend on the willingness of the patients, but is also influenced by external factors. Raising awareness of SAD among general practitioners and low-level access to mental-health support could help patients find appropriate help sooner. To better guide the optimal treatment choice, comparative effectiveness research on treatments to prevent a new onset in patients with a history of SAD and clinical practice guidelines on SAD are needed.

Abstract Background Hip fractures are a major public health problem in elderly populations and are accompanied by high-mortality rates. Whether timing of surgery has an impact on morbidity and mortality has been discussed controversially, numerous studies suggest that the delay of surgery can significantly increase the risk of morbidity and mortality; others report that achieving a stable medical condition is more important than early surgery. The goal of our systematic review is to assess the impact of timing of surgery on health outcomes in patients aged 60 years or older with acute hip fracture. In addition, we will investigate differences in beneficial or harmful effects of timing of surgery in subgroups of patients based on demographic characteristics, physical status, and the use of anticoagulant medications. Methods We will systematically search MEDLINE via Ovid, the Cochrane Library, Embase, PubMed, and clinical trial registries (from 1997 to 2017). In addition, we will search reference lists of pertinent reviews, archives of annual meetings of orthopaedic societies, and contact experts. We will include randomized controlled trials and non-randomized studies assessing the impact of timing of surgery after hip fracture in patients 60 years or older, published in English or German. Our outcomes of interest include health outcomes such as mortality, perioperative complications, functional capacity, and quality of life. We plan to perform meta-analyses if we have at least three sufficiently similar studies. If data are sufficient, we will conduct subgroup-analyses testing for differences between age groups, sex, patients’ physical status as assessed with ASA (American Society of Anesthesiologists) scores, and the use of anticoagulation. Discussion Since this is the first systematic review on this topic since 2010, our findings will help to inform clinical practice guidelines concerning timing of surgery in hip fractures. Furthermore, our findings could contribute to define an optimal time period for surgery for different groups of patients with acute hip fracture. Systematic review registration PROSPERO 2017 CRD42017058216

Abstract Background Decisionmakers and guideline developers demand rapid syntheses of the evidence when time sensitive evidence-informed decisions are required. A potential trade-off of such rapid reviews is that their results can have less reliability than results of systematic reviews that can lead to an increased risk of making incorrect decisions or recommendations. We sought to determine how much incremental uncertainty about the correctness of an answer guideline developers and health policy decisionmakers are willing to accept in exchange for a rapid evidence-synthesis. Methods Employing a purposive sample, we conducted an international web-based, anonymous survey of decisionmakers and guideline developers. Based on a clinical treatment, a public health, and a clinical prevention scenario, participants indicated the maximum risk of getting an incorrect answer from a rapid review that they would be willing to accept. We carefully reviewed data and performed descriptive statistical analyses. Results In total, 325 (58.5%) of 556 participants completed our survey and were eligible for analysis. The median acceptable incremental risk for getting an incorrect answer from a rapid review across all three scenarios was 10.0% (interquartile range [IQR] 5.0–15.0). Acceptable risks were similar for the clinical treatment (n = 313, median 10.0% [IQR 5.0–15.0]) and the public health scenarios (n = 320, median 10.0% [IQR 5.0–15.0]) and lower for the clinical prevention scenario (n = 312, median 6.5% [IQR 5.0–10.5]). Conclusions Findings suggest that decisionmakers are willing to accept some trade-off in validity in exchange for a rapid review. Nevertheless, they expect the validity of rapid reviews to come close to that of systematic reviews.

Abstract Background Systematic reviews offer the most reliable and valid support for health policy decision-making, patient information, and guideline development. However, they are labor intensive and frequently take longer than 1 year to complete. Consequently, they often do not meet the needs of those who need to make decisions quickly. Rapid reviews have therefore become a pragmatic alternative to systematic reviews. They are knowledge syntheses that abbreviate certain methodological aspects of systematic reviews to produce information more quickly. Methodological shortcuts often take place in literature identification. A potential drawback is less reliable results. To date, the impact of abbreviated searches on estimates of treatment effects and subsequent conclusions has not been analyzed systematically across multiple bodies of evidence. We aim to answer the research question: Do bodies of evidence that are based on abbreviated literature searches lead to different conclusions about benefits and harms of interventions compared with bodies of evidence that are based on comprehensive, systematic literature searches? Methods We will use a non-inferiority and meta-epidemiologic design. The primary outcome is the proportion of discordant conclusions based on different search approaches. Drawing of a pool of Cochrane reports published between 2012 and 2016, we will randomly select 60 reports. Eligible reports are those that present a summary-of-findings table, draw a clear conclusion, present data for meta-analyses, and document the search strategy clearly. We will conduct several abbreviated searches to detect whether included studies in these Cochrane reviews could be detected. If searches could not detect all studies, we will revise the original summary-of-findings table and ask review authors whether the missed evidence would change conclusions of their report. We will determine the proportion of discordant conclusions for each abbreviated search approach. We will consider an abbreviated search as non-inferior if the lower limit of the 95% confidence interval of the proportion of discordant conclusions is below the non-inferiority margin, which is determined based on results of a survey for clinical and public health scenarios. Discussion This will be the first study to assess whether the reduced sensitivity of abbreviated searches has an impact on conclusions across multiple bodies of evidence, not only on effect estimates.