Search

Background: Utilisation of crowdsourcing within evidence synthesis has increased over the last decade. Crowdsourcing platform Cochrane Crowd has engaged a global community of 22,000 people from 170 countries. The COVID-19 pandemic presented an opportunity to engage the community and keep up with the exponential output of COVID-19 research. Aims: To test whether a crowd could accurately assess study eligibility for reviews under time constraints. Outcome measures: time taken to complete each task, time to produce required training modules, crowd sensitivity, specificity and crowd consensus. Methods: We created four crowd tasks, corresponding to four Cochrane COVID-19 Rapid Reviews. The search results of each were uploaded and an interactive training module was developed for each task. Contributors who had participated in another COVID-19 task were invited to participate. Each task was live for 48-h. The final inclusion and exclusion decisions made by the core author team were used as the reference standard. Results: Across all four reviews 14,299 records were screened by 101 crowd contributors. The crowd completed each screening task within 48-h for three reviews and in 52 h for one. Sensitivity ranged from 94% to 100%. Four studies, out of a total of 109, were incorrectly rejected by the crowd. However, their absence ultimately would not have altered the conclusions of the reviews. Crowd consensus ranged from 71% to 92% across the four reviews. Conclusion: Crowdsourcing can play a valuable role in study identification and offers willing contributors the opportunity to help identify COVID-19 research for rapid evidence syntheses.

Comprehensive systematic reviews require many hours of manual review by researchers. This study retrospectively investigates the use of GPT-3.5 Turbo to screen titles and abstracts as a second reviewer or to prescreen citations for human review. The authors describe the process of prompt development as well as the performance of GPT-3.5 Turbo in terms of citations missed and the added number of citations for reconciliation for 5 different already completed reviews. Background: Systematic reviews are performed manually despite the exponential growth of scientific literature. Objective: To investigate the sensitivity and specificity of GPT-3.5 Turbo, from OpenAI, as a single reviewer, for title and abstract screening in systematic reviews. Design: Diagnostic test accuracy study. Setting: Unannotated bibliographic databases from 5 systematic reviews representing 22 665 citations. Participants: None. Measurements: A generic prompt framework to instruct GPT to perform title and abstract screening was designed. The output of the model was compared with decisions from authors under 2 rules. The first rule balanced sensitivity and specificity, for example, to act as a second reviewer. The second rule optimized sensitivity, for example, to reduce the number of citations to be manually screened. Results: Under the balanced rule, sensitivities ranged from 81.1% to 96.5% and specificities ranged from 25.8% to 80.4%. Across all reviews, GPT identified 7 of 708 citations (1%) missed by humans that should have been included after full-text screening at the cost of 10 279 of 22 665 false-positive recommendations (45.3%) that would require reconciliation during the screening process. Under the sensitive rule, sensitivities ranged from 94.6% to 99.8% and specificities ranged from 2.2% to 46.6%. Limiting manual screening to citations not ruled out by GPT could reduce the number of citations to screen from 127 of 6334 (2%) to 1851 of 4077 (45.4%), at the cost of missing from 0 to 1 of 26 citations (3.8%) at the full-text level. Limitations: Time needed to fine-tune prompt. Retrospective nature of the study, convenient sample of 5 systematic reviews, and GPT performance sensitive to prompt development and time. Conclusion: The GPT-3.5 Turbo model may be used as a second reviewer for title and abstract screening, at the cost of additional work to reconcile added false positives. It also showed potential to reduce the number of citations before screening by humans, at the cost of missing some citations at the full-text level. Primary Funding Source: None. [ABSTRACT FROM AUTHOR]

Summary: Importance: The effectiveness of anti‐obesity medications for children and adolescents is unclear. Objective: To update the evidence on the benefits and harms of anti‐obesity medication. Data Sources: Cochrane CENTRAL, MEDLINE, ClinicalTrials.gov and WHO ICTRP (1/1/16–17/3/23). Study Selection: Randomized controlled trials ≥6 months in people <19 years living with obesity. Data Extraction and Synthesis: Screening, data extraction and quality assessment conducted in duplicate, independently. Main Outcomes and Measures: Body mass index (BMI): 95th percentile BMI, adverse events and quality of life. Results: Thirty‐five trials (N = 4331), follow‐up: 6–24 months; age: 8.8–16.3 years; BMI: 26.2–41.7 kg/m2. Moderate certainty evidence demonstrated a −1.71 (95% confidence interval [CI]: −2.27 to −1.14)‐unit BMI reduction, ranging from −0.8 to −5.9 units between individual drugs with semaglutide producing the largest reduction of −5.88 kg/m2 (95% CI: −6.99 to −4.77, N = 201). Drug type explained ~44% of heterogeneity. Low certainty evidence demonstrated reduction in 95th percentile BMI: −11.88 percentage points (95% CI: −18.43 to −5.30, N = 668). Serious adverse events and study discontinuation due to adverse events did not differ between medications and comparators, but medication dose adjustments were higher compared to comparator (10.6% vs 1.7%; RR = 3.74 [95% CI: 1.51 to 9.26], I2 = 15%), regardless of approval status. There was a trend towards improved quality of life. Evidence gaps exist for children, psychosocial outcomes, comorbidities and weight loss maintenance. Conclusions and Relevance: Anti‐obesity medications in addition to behaviour change improve BMI but may require dose adjustment, with 1 in 100 adolescents experiencing a serious adverse event. [ABSTRACT FROM AUTHOR]