Search

Throughout the global coronavirus pandemic, we have seen an unprecedented volume of COVID-19 researchpublications. This vast body of evidence continues to grow, making it difficult for research users to keep up with the pace of evolving research findings. To enable the synthesis of this evidence for timely use by researchers, policymakers, and other stakeholders, we developed an automated workflow to collect, categorise, and visualise the evidence from primary COVID-19 research studies. We trained a crowd of volunteer reviewers to annotate studies by relevance to COVID-19, study objectives, and methodological approaches. Using these human decisions, we are training machine learning classifiers and applying text-mining tools to continually categorise the findings and evaluate the quality of COVID-19 evidence.

BackgroundResearchers who perform systematic searches across multiple databases often identify duplicate publications. Identifying such duplicates (“deduplication”) can be extremely time-consuming, but failure to remove these citations can, in the worst instance, lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews.MethodsWe evaluated ASySD’s performance on 5 unseen biomedical systematic search datasets of various sizes (1,845 – 79,880 citations), which had been deduplicated by human reviewers. We compared the performance of ASySD with Endnote’s automated deduplication option and with the Systematic Review Accelerator Deduplication Module (SRA-DM).ResultsASySD identified more duplicates than either SRA-DM or Endnote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of 0.94-0.99. The tool took less than 1 hour to deduplicate all datasets.ConclusionsFor duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.

Preclinical research is a vital step in the drug discovery pipeline and more generally in helping to better understand human disease aetiology and its management. Systematic reviews (SRs) can be powerful in summarising and appraising this evidence concerning a specific research question, to highlight areas of improvements, areas for further research and areas where evidence may be sufficient to take forward to other research domains, for instance clinical trial. Guidance and tools for preclinical research synthesis remain limited despite their clear utility. We aimed to create an online end-to-end platform primarily for conducting SRs of preclinical studies, that was flexible enough to support a wide variety of experimental designs, was adaptable to different research questions, would allow users to adopt emerging automated tools and support them during their review process using best practice. In this article, we introduce the Systematic Review Facility (https://syrf.org.uk), which was launched in 2016 and designed to support primarily preclinical SRs from small independent projects to large, crowdsourced projects. We discuss the architecture of the app and its features, including the opportunity to collaborate easily, to efficiently manage projects, to screen and annotate studies for important features (metadata), to extract outcome data into a secure database, and tailor these steps to each project. We introduce how we are working to leverage the use of automation tools and allow the integration of these services to accelerate and automate steps in the systematic review workflow.

BackgroundThe reporting of the measures taken to reduce risks of bias in in vivo research provides important information to research users and represents an important aspect of research quality. The testing of the effectiveness of interventions to improve research quality, and efforts in research improvement, both require the timely and efficient assessment of performance. MethodsWe describe methods which will allow (i) Identification of reports of in vivo research, (ii) retrieval of full text; (iii) attribution of journal, institution, and funder; (iv) automated annotation of the reporting of measures to reduce risks of bias, aggregated by year, journal, institution and funder.DiscussionThis observational study will provide information to guide efforts in research improvement.

Metaresearch is a scientific field involving the study of research itself. It has been applied to clinical trials since the 1980s,1 but has only become an emerging discipline over the last decade in the preclinical field. The primary tool of metaresearch is the systematic review, which uses predefined methods to provide a transparent and comprehensive summary of the evidence relating to a research question. A systematic review is defined as ‘a review that uses explicit, systematic methods to collate and synthesize findings of studies that address a clearly formulated question’.2 Systematic reviews allow for evaluation of methods and comprehensiveness of reporting, to assay likelihood of reproducibility and potential for translatability to subsequent domains of research and can investigate the impact of incentives on primary research. This, in turn, allows for a more rigorous understanding of what makes research reliable, and how research can be improved,3 while driving evidence-based decisions for future research.4 5 Systematic reviews typically comprise several steps. Before beginning a systematic review, it is recommended that the author team develop a protocol, which defines the research question and the methods that will be used to conduct, analyse and report the findings of the review. The research question determines the resources required to complete the review, the broader the question and the larger the field the more resource intensive a review will be. The search strategy is developed to identify as much potentially relevant literature as possible, often involving searching of multiple databases. Following database searches, deduplication (if searching multiple database with overlapping coverage), the unique search results are screened for inclusion or exclusion. Full-text retrieval may be conducted before or after title and abstract citation screening. Metadata including information regarding reported study quality and design, and outcome data are then extracted from the included …

An important factor for successful translational stroke research is study quality. Low-quality studies are at risk of biased results and effect overestimation, as has been intensely discussed for small animal stroke research. However, little is known about the methodological rigor and quality in large animal stroke models, which are becoming more frequently used in the field. Based on research in two databases, this systematic review surveys and analyses the methodological quality in large animal stroke research. Quality analysis was based on the Stroke Therapy Academic Industry Roundtable and the Animals in Research: Reporting In Vivo Experiments guidelines. Our analysis revealed that large animal models are utilized with similar shortcomings as small animal models. Moreover, translational benefits of large animal models may be limited due to lacking implementation of important quality criteria such as randomization, allocation concealment, and blinded assessment of outcome. On the other hand, an increase of study quality over time and a positive correlation between study quality and journal impact factor were identified. Based on the obtained findings, we derive recommendations for optimal study planning, conducting, and data analysis/reporting when using large animal stroke models to fully benefit from the translational advantages offered by these models.

Carnosine is a naturally occurring pleotropic dipeptide which influences multiple deleterious mechanisms that are\ud activated during stroke. Numerous published studies have reported that carnosine has robust efficacy in ischemic\ud stroke models. To further evaluate these data, we have conducted a systematic review and meta-analysis of published\ud studies. We included publications describing in vivo models of ischemic stroke where the neuroprotective efficacy of\ud carnosine was being evaluated through the reporting of infarct volume and/or neurological score as outcomes. Overall\ud efficacy was evaluated using weighted mean difference random effects meta-analysis. We also evaluated for study quality\ud and publication bias. We identified eight publications that met our inclusion criteria describing a total of 29 comparisons\ud and 454 animals. Overall methodological quality of studies was moderate (median ¼ 4/9). Carnosine reduced infarct\ud volume by 29.4% (95% confidence interval (CI), 24.0% to 34.9%; 29 comparisons). A clear dose-response effect was\ud observed, and efficacy was reduced when carnosine was administered more than 6 h after ischemia. Our findings suggest\ud that carnosine administered before or after the onset of ischemia exhibits robust efficacy in experimental ischemic\ud stroke. However, the methodological quality of some of the studies was low and testing occurred only in healthy young\ud male animals.

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality., A systematic analysis of in vivo research reveals poor reporting of measures that reduce the risk of bias and an inverse relationship between impact factor and the reporting of randomization.