Application of artificial intelligence in clinical diagnosis and treatment: an overview of systematic reviews

Objective: This study aimed to summarize the characteristics and methodological quality of systematic reviews on the application of artificial intelligence (AI) in clinical diagnosis and treatment. Methods: We systematically searched seven English- and Chinese-language literature databases to identify systematic reviews on the application of AI, deep learning, or machine learning in the diagnosis and treatment of any disease published in 2020. We evaluated the methodological quality of the included systematic reviews using “A Measurement tool for the assessment of multiple systematic reviews” (AMSTAR). We also conducted meta-analyses on the diagnostic accuracy of AI on selected disease categories with a large number of included studies and low clinical heterogeneity. Results: A total of 40 systematic reviews reporting 1,083 original studies were included, covering 31 diseases from 11 groups of diseases. Eleven systematic reviews were related to neoplasms and nine were systematic reviews related to diseases of the digestive system. We selected digestive system diseases for the meta-analysis. The pooled sensitivities (with 95% confidence interval (CI)) of AI to assist the diagnosis of helicobacter pylori, gastrointestinal ulcers, hemorrhage, esophageal tumors, gastric tumors, and intestinal tumors (with 95% CI) were 0.91 (0.83–0.95), 0.99 (0.76–1.00), 0.95 (0.83–0.99), 0.90 (0.85–0.93), 0.90 (0.82–0.95), and 0.93 (0.88–0.96), respectively, and the pooled specificities were 0.82 (0.77–0.87), 0.97 (0.86–1.00), 1.00 (0.99–1.00), 0.80 (0.71–0.87), 0.93 (0.87–0.97), and 0.89 (0.85–0.92), respectively. The AMSTAR items “the list of included studies” (n = 39, 97.5%) and “the characteristics of the included studies” (n = 39, 97.5%) had the highest compliance among the reviews; the compliance was relatively low to the items “the consideration of publication status” (n = 1, 2.5%), “the consideration of scientific quality” (n = 19, 47.5%), “data synthesis methods” (n = 18, 45.0%), and “ the evaluation of publication bias” (n = 13, 32.5%). Conclusions: The main subjects of systematic reviews on AI applications in clinical diagnosis and treatment published in 2020 were diseases of the digestive system and neoplasms. The methodological quality of the systematic reviews on AI needs to be improved, paying particular attention to publication bias and the rigorous evaluation of the quality of the included studies.