Search

Background The Cochrane Bias Methods Group recently developed the “Risk of Bias (ROB) in Non-randomized Studies of Interventions” (ROBINS-I) tool to assess ROB for non-randomized studies of interventions (NRSI). It is important to establish consistency in its application and interpretation across review teams. In addition, it is important to understand if specialized training and guidance will improve the reliability of the results of the assessments. Therefore, the objective of this cross-sectional study is to establish the inter-rater reliability (IRR), inter-consensus reliability (ICR), and concurrent validity of ROBINS-I. Furthermore, as this is a relatively new tool, it is important to understand the barriers to using this tool (e.g., time to conduct assessments and reach consensus—evaluator burden). Methods Reviewers from four participating centers will appraise the ROB of a sample of NRSI publications using the ROBINS-I tool in two stages. For IRR and ICR, two pairs of reviewers will assess the ROB for each NRSI publication. In the first stage, reviewers will assess the ROB without any formal guidance. In the second stage, reviewers will be provided customized training and guidance. At each stage, each pair of reviewers will resolve conflicts and arrive at a consensus. To calculate the IRR and ICR, we will use Gwet’s AC1 statistic. For concurrent validity, reviewers will appraise a sample of NRSI publications using both the New-castle Ottawa Scale (NOS) and ROBINS-I. We will analyze the concordance between the two tools for similar domains and for the overall judgments using Kendall’s tau coefficient. To measure the evaluator burden, we will assess the time taken to apply the ROBINS-I (without and with guidance), and the NOS. To assess the impact of customized training and guidance on the evaluator burden, we will use the generalized linear models. We will use Microsoft Excel and SAS 9.4 to manage and analyze study data, respectively. Discussion The quality of evidence from systematic reviews that include NRS depends partly on the study-level ROB assessments. The findings of this study will contribute to an improved understanding of the ROBINS-I tool and how best to use it.

ObjectivesTo summarise the current evidence regarding interventions for accurate and timely cancer diagnosis among symptomatic individuals.DesignA scoping review following the Joanna Briggs Institute’s methodological framework for the conduct of scoping reviews and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews checklist.Data sourcesMEDLINE (Ovid), CINAHL (EBSCOhost) and PsycINFO (Ovid) bibliographic databases, and websites of relevant organisations. Published and unpublished literature (grey literature) of any study type in the English language were searched for from January 2017 to January 2021.Eligibility and criteriaStudy participants were individuals of any age presenting at clinics with symptoms indicative of cancer. Interventions included practice guidelines, care pathways or other initiatives focused on achieving predefined benchmarks or targets for wait times, streamlined or rapid cancer diagnostic services, multidisciplinary teams and patient navigation strategies. Outcomes included accuracy and timeliness of cancer diagnosis.Data extraction and synthesisWe summarised findings graphically and descriptively.ResultsFrom 21 298 retrieved citations, 88 unique published articles and 16 unique unpublished documents (on 18 study reports), met the eligibility for inclusion. About half of the published literature and 83% of the unpublished literature were from the UK. Most of the studies were on interventions in patients with lung cancer. Rapid referral pathways and technology for supporting and streamlining the cancer diagnosis process were the most studied interventions. Interventions were mostly complex and organisation-specific. Common themes among the studies that concluded intervention was effective were multidisciplinary collaboration and the use of a nurse navigator.ConclusionsMultidisciplinary cooperation and involvement of a nurse navigator may be unique features to consider when designing, delivering and evaluating interventions focused on improving accurate and timely cancer diagnosis among symptomatic individuals. Future research should examine the effectiveness of the interventions identified through this review.

ObjectivesTo conduct a scoping review to identify and summarise the existing literature on interventions involving primary healthcare professionals to manage emergency department (ED) overcrowding.DesignA scoping review.Data sourcesA comprehensive database search of Medline (Ovid), EMBASE (Ovid), Cochrane Library (Wiley) and CINAHL (EBSCO) databases was conducted (inception until January 2020) using peer-reviewed search strategies, complemented by a search of grey literature sources.Eligibility criteriaInterventions and strategies involving primary healthcare professionals (PHCPs: general practitioners (GPs), nurse practitioners (NPs) or nurses with expanded role) to manage ED overcrowding.MethodsWe engaged and collaborated, with 13 patient partners during the design and conduct stages of this review. We conducted this review using the JBI guidelines. Two reviewers independently selected studies and extracted data. We conducted descriptive analysis of the included studies (frequencies and percentages).ResultsFrom 23 947 records identified, we included 268 studies published between 1981 and 2020. The majority (58%) of studies were conducted in North America and were predominantly cohort studies (42%). The reported interventions were either ‘within ED’ (48%) interventions (eg, PHCP-led ED triage or fast track) or ‘outside ED’ interventions (52%) (eg, after-hours GP clinic and GP cooperatives). PHCPs involved in the interventions were: GP (32%), NP (26%), nurses with expanded role (16%) and combinations of the PHCPs (42%). The ‘within ED’ and ‘outside ED’ interventions reported outcomes on patient flow and ED utilisation, respectively.ConclusionsWe identified many interventions involving PHCPs that predominantly reported a positive impact on ED utilisation/patient flow metrics. Future research needs to focus on conducting well-designed randomized controlled trials (RCTs) and systematic reviews to evaluate the effectiveness of specific interventions involving PHCPs to critically appraise and summarise evidence on this topic.

In view of many unanswered clinical questions regarding treatment of COVID-19 with remdesivir, we systematically identified, critically appraised and summarized the findings from randomized controlled trials (RCTs) of remdesivir for COVID-19. We searched relevant databases/websites (up to September 2020) and selected English-language RCT publications of remdesivir for COVID-19. We conducted meta-analysis using an inverse variance, random-effects model in addition to trial sequential analysis (TSA) for the efficacy outcomes: all-cause mortality, viral burden and clinical progression. Safety outcomes were diarrhoea, nausea, and vomiting. We calculated the relative risk (RR) and 95% confidence interval (CI) for all outcomes. Statistical heterogeneity was calculated using the I2 statistic. We included five RCTs (7540 participants) from 7237 citations. Most (80%) were of an unclear to high risk of bias. There was no evidence of a significant improvement with remdesivir (100 mg, 10 days) regarding all-cause mortality (RR 0.94, CI 0.82–1.07; I2 = 0%; 4 RCTs; 7143 patients), clinical progression (RR 1.08, CI 0.99–1.18; I2 = 70.4%; 3 RCTs; 1692 patients), or diarrhoea (RR 0.82, CI 0.40–1.66; I2 = 0%; 2 RCTs; 630 patients). Nausea occurred more often with remdesivir (RR 2.77, CI 1.28–6.03; I2 = 0%; 2 RCTs; 630 patients). TSA showed that the required information size was not reached for firm conclusions to be drawn. There is insufficient evidence to support the use of remdesivir for treatment of COVID-19. More high-quality RCTs are needed for a stronger evidence. Until then, remdesivir should remain an experimental drug for COVID-19.

Objectives To evaluate the impact of guidance and training on the inter-rater reliability (IRR), inter-consensus reliability (ICR) and evaluator burden of the Risk of Bias (RoB) in Non-randomized Studies (NRS) of Interventions (ROBINS-I) tool, and the RoB instrument for NRS of Exposures (ROB-NRSE). Study design and Setting In a before-and-after study, seven reviewers appraised the RoB using ROBINS-I (n = 44) and ROB-NRSE (n = 44), before and after guidance and training. We used Gwet's AC1 statistic to calculate IRR and ICR. Results After guidance and training, the IRR and ICR of the overall bias domain of ROBINS-I and ROB-NRSE improved significantly; with many individual domains showing either a significant (IRR and ICR of ROB-NRSE; ICR of ROBINS-I), or nonsignificant improvement (IRR of ROBINS-I). Evaluator burden significantly decreased after guidance and training for ROBINS-I, whereas for ROB-NRSE there was a slight nonsignificant increase. Conclusion Overall, there was benefit for guidance and training for both tools. We highly recommend guidance and training to reviewers prior to RoB assessments and that future research investigate aspects of guidance and training that are most effective.

Objective To assess the real-world interrater reliability (IRR), interconsensus reliability (ICR), and evaluator burden of the Risk of Bias (RoB) in Nonrandomized Studies (NRS) of Interventions (ROBINS-I), and the ROB Instrument for NRS of Exposures (ROB-NRSE) tools. Study Design and Setting A six-center cross-sectional study with seven reviewers (2 reviewer pairs) assessing the RoB using ROBINS-I (n = 44 NRS) or ROB-NRSE (n = 44 NRS). We used Gwet’s AC1 statistic to calculate the IRR and ICR. To measure the evaluator burden, we assessed the total time taken to apply the tool and reach a consensus. Results For ROBINS-I, both IRR and ICR for individual domains ranged from poor to substantial agreement. IRR and ICR on overall RoB were poor. The evaluator burden was 48.45 min (95% CI 45.61 to 51.29). For ROB-NRSE, the IRR and ICR for the majority of domains were poor, while the rest ranged from fair to perfect agreement. IRR and ICR on overall RoB were slight and poor, respectively. The evaluator burden was 36.98 min (95% CI 34.80 to 39.16). Conclusions We found both tools to have low reliability, although ROBINS-I was slightly higher. Measures to increase agreement between raters (e.g., detailed training, supportive guidance material) may improve reliability and decrease evaluator burden.

Background A new tool, “risk of bias (ROB) instrument for non-randomized studies of exposures (ROB-NRSE),” was recently developed. It is important to establish consistency in its application and interpretation across review teams. In addition, it is important to understand if specialized training and guidance will improve the reliability in the results of the assessments. Therefore, the objective of this cross-sectional study is to establish the inter-rater reliability (IRR), inter-consensus reliability (ICR), and concurrent validity of the new ROB-NRSE tool. Furthermore, as this is a relatively new tool, it is important to understand the barriers to using this tool (e.g., time to conduct assessments and reach consensus—evaluator burden). Methods Reviewers from four participating centers will apprise the ROB of a sample of NRSE publications using ROB-NRSE tool in two stages. For IRR and ICR, two pairs of reviewers will assess the ROB for each NRSE publication. In the first stage, reviewers will assess the ROB without any formal guidance. In the second stage, reviewers will be provided customized training and guidance. At each stage, each pair of reviewers will resolve conflicts and arrive at a consensus. To calculate the IRR and ICR, we will use Gwet’s AC1 statistic. For concurrent validity, reviewers will appraise a sample of NRSE publications using both the Newcastle-Ottawa Scale (NOS) and ROB-NRSE tool. We will analyze the concordance between the two tools for similar domains and for the overall judgments using Kendall’s tau coefficient. To measure evaluator burden, we will assess the time taken to apply ROB-NRSE tool (without and with guidance), and the NOS. To assess the impact of customized training and guidance on the evaluator burden, we will use the generalized linear models. We will use Microsoft Excel and SAS 9.4, to manage and analyze study data, respectively. Discussion The quality of evidence from systematic reviews that include NRSE depends partly on the study-level ROB assessments. The findings of this study will contribute to an improved understanding of ROB-NRSE and how best to use it.

Background Delays in reperfusion for patients with myocardial ischemia leads to increased morbidity and mortality. The objective of this review was to identify, evaluate, and critically appraise the evidence on whether pre-hospital electrocardiography (ECG) reduces patient mortality and improves post—ST-segment myocardial infarction patient-oriented outcomes. Methods We searched PubMed/MEDLINE, EMBASE, and Cochrane Library (1990-2015) for controlled clinical studies. We also searched conference proceedings, trial registries, and reference lists of narrative and systematic reviews. Two reviewers independently identified and extracted data from studies that compared pre-hospital ECG with standard of care in patients with suspected myocardial infarction who underwent primary percutaneous coronary intervention. Internal validity was assessed using the Newcastle-Ottawa scale. Results We screened 21,197 citations and included 63 unique studies (plus 22 companion publications). Most studies were of moderate quality. Pre-hospital ECG was associated with significantly fewer deaths (relative risk, 0.68; 95% confidence interval [CI], 0.63-0.74; 45 studies; 71,315 patients; I 2 , 0%), reduced time to reperfusion (mean difference, −35.32 minutes; 95% CI, −44.02 to −26.61; 26 studies; 27,524 patients; I 2 , 97%), shorter hospital stays (mean difference, −0.63 days; 95% CI, −1.05 to −0.20; 10 studies; 39,275 patients; I 2 , 39%), and more patients had first medical contact to device time I 2 , 93%). Conclusions Use of pre-hospital ECG is associated with decreased mortality and overall better patient outcomes.

Background Randomized controlled trials (RCTs) on bone marrow stem cell (BMSC) therapy in ST-elevation myocardial infarction (STEMI) patients have reported conflicting results. Our main objective was to critically appraise and meta-analyze best-available evidence on efficacy and safety of intracoronary administration of autologous BMSC therapy in STEMI patients after primary percutaneous coronary intervention. Methods We conducted a search of MEDLINE, PubMed, EMBASE, CENTRAL, Global Health, CINAHL, and conference proceedings in February 2017. Our primary outcome was all-cause mortality. Secondary and safety outcomes included cardiac death, heart failure, arrhythmias, repeat myocardial infarction, or target vessel revascularizations; or improved health-related quality of life, left ventricular ejection fraction, or infarct size. Summary relative and absolute risks were obtained using random effects models. We also evaluated the strength of evidence. Results A comprehensive database search identified 42 RCTs (3365 STEMI patients). BMSC therapy did not significantly decrease mortality (risk ratio, 0.71; 95% confidence interval, 0.45-1.11; I 2 , 0%; absolute risk reduction, 0.1%; 95% confidence interval, −0.71 to 0.91; 40 trials; 3289 participants; I 2 , 0%; low strength of evidence). BMSC therapy had no effect on secondary or adverse outcomes. Trial sequential analysis for all-cause mortality showed no evidence of a clinically important difference, with a very low probability that future studies can change the current conclusion. Conclusions On the basis of evidence from 42 RCTs published in the past 15 years, we provide conclusive evidence for a lack of beneficial effect for autologous BMSC therapy in patients with STEMI.

CONTEXT: Nonnutritive sweetener (NNS) consumption is increasing among children, yet its long-term health impact is unclear, particularly when exposure occurs during early life. OBJECTIVE: To synthesize evidence from prospective studies evaluating the association of early-life NNS exposure and long-term metabolic health. DATA SOURCES: Medline, Embase, and Cochrane Library (inception to July 2015). STUDY SELECTION: We aimed to include randomized controlled trials (RCTs) evaluating NNS-based interventions and prospective cohort studies reporting NNS exposure among pregnant women, infants, or children ( DATA EXTRACTION: The primary outcome was BMI; secondary outcomes included growth velocity, overweight/obesity, adiposity, and adverse metabolic effects. Study quality and risk of bias were evaluated using validated assessment tools. RESULTS: We identified 6 eligible cohort studies and 2 RCTs (n = 15 641 children). Half of the cohorts reported increasing weight gain or fat mass accumulation with increasing NNS intake, and pooled data from 2 cohorts showed a significant correlation with BMI gain (weighted mean correlation 0.023, 95% confidence interval 0.006 to 0.041). RCTs reported contradictory effects on weight change in children receiving NNSs. No eligible studies evaluated prenatal or infant NNS exposure. LIMITATIONS: Meta-analysis was limited because of the small number of eligible studies and heterogeneity of populations and outcomes. CONCLUSIONS: There is limited and inconsistent evidence of the long-term metabolic effects of NNS exposure during gestation, infancy, and childhood. Further research is needed to inform recommendations for the use of NNSs in this sensitive population.