Search

Abstract Background The COVID-19 pandemic has created a sense of urgency in the research community in their bid to contribute to the evidence required for healthcare policy decisions. With such urgency, researchers experience methodological challenges to maintain the rigour and transparency of their work. With this in mind, we offer reflections on our recent experience of undertaking a rapid Cochrane qualitative evidence synthesis (QES). Methods This process paper, using a reflexive approach, describes a rapid QES prepared during, and in response to, the COVID-19 pandemic. Findings This paper reports the methodological decisions we made and the process we undertook. We place our decisions in the context of guidance offered in relation to rapid reviews and previously conducted QESs. We highlight some of the challenges we encountered in finding the balance between the time needed for thoughtfulness and comprehensiveness whilst providing a rapid response to an urgent request for evidence. Conclusion The need for more guidance on rapid QES remains, but such guidance needs to be based on actual worked examples and case studies. This paper and the reflections offered may provide a useful framework for others to use and further develop.

Abstract Background Systematic reviews of randomised trials guide policy and healthcare decisions. Yet, we observed that some reviews judge randomised trials as high or unclear risk of bias (ROB) for sequence generation, potentially introducing bias. However, to date, the extent of this issue has not been well examined. We evaluated the consistency in the ROB assessment for sequence generation of randomised trials in Cochrane and non-Cochrane reviews, and explored the reviewers’ judgement of the quality of evidence for the related outcomes. Methods Cochrane intervention reviews (01/01/2017–31/03/2017) were retrieved from the Cochrane Database of Systematic Reviews. We also searched for systematic reviews in ten general medical journals with highest impact factors (01/01/2016–31/03/2017). We examined the proportion of reviews that rated the sequence generation domain as high, low or unclear risk of selection bias. For reviews that had rated any randomised trials as high or unclear risk of bias, we examined the proportion that had assessed the quality of evidence. Results Overall, 100 systematic reviews were included in our analysis. We evaluated 64 Cochrane reviews which comprised of 984 randomised trials; 0.8% (n = 8) and 52.2% (n = 514) were rated as high and unclear ROB for sequence generation respectively. We further evaluated 36 non-Cochrane reviews which comprised of 1376 trials; 5.8% (n = 80) and 39.6% (n = 545) were rated as high and unclear ROB respectively. Ninety percent (n = 10) of non-Cochrane reviews which rated randomised trials as high ROB for sequence generation did not report an underlying reason. All Cochrane reviews assessed the quality of evidence (GRADE). For the non-Cochrane reviews, only just over half had assessed the quality of evidence. Conclusion Systematic reviews of interventions frequently rate randomised trials as high or unclear ROB for sequence generation. In general, Cochrane reviews were more transparent than non-Cochrane reviews in ROB and quality of evidence assessment. The scientific community should more strongly promote consistent ROB assessment for sequence generation to minimise selection bias and support transparent quality of evidence assessment. Consistency ensures that appropriate conclusions are drawn from the data.