1. A machine-learning algorithm for diagnosis of multisystem inflammatory syndrome in children and Kawasaki disease in the USA: a retrospective model development and validation study
- Author
-
Lam, Jonathan Y, Shimizu, Chisato, Tremoulet, Adriana H, Bainto, Emelia, Roberts, Samantha C, Sivilay, Nipha, Gardiner, Michael A, Kanegaye, John T, Hogan, Alexander H, Salazar, Juan C, Mohandas, Sindhu, Szmuszkovicz, Jacqueline R, Mahanta, Simran, Dionne, Audrey, Newburger, Jane W, Ansusinha, Emily, DeBiasi, Roberta L, Hao, Shiying, Ling, Xuefeng B, Cohen, Harvey J, Nemati, Shamim, Burns, Jane C, Group, Pediatric Emergency Medicine Kawasaki Disease Research, Abe, Naomi, Austin-Page, Lukas R, Bryl, Amy W, Donofrio-Odmann, J Joelle, Ekpenyong, Atim, Gutglass, David J, Nguyen, Margaret B, Schwartz, Kristy, Ulrich, Stacey, Vayngortin, Tatyana, Zimmerman, Elise, Group, CHARMS Study, Anderson, Marsha, Ang, Jocelyn Y, Ashouri, Negar, Bocchini, Joseph, D'Addese, Laura, Dominguez, Samuel, Gutierrez, Maria Pila, Harahsheh, Ashraf S, Hite, Michelle, Jone, Pei-Ni, Kumar, Madan, Manaloor, John J, Melish, Marian, Morgan, Lerraughn, Natale, JoAnne E, Rometo, Allison, Rosenkranz, Margalit, Rowley, Anne H, Samuy, Nichole, Scalici, Paul, and Sykes, Michelle
- Subjects
Health Services and Systems ,Health Sciences ,Patient Safety ,Prevention ,Rare Diseases ,Clinical Research ,Pediatric ,4.2 Evaluation of markers and technologies ,Detection ,screening and diagnosis ,Algorithms ,Artificial Intelligence ,COVID-19 ,COVID-19 Testing ,Child ,Humans ,Machine Learning ,Mucocutaneous Lymph Node Syndrome ,Pandemics ,Retrospective Studies ,SARS-CoV-2 ,Systemic Inflammatory Response Syndrome ,United States ,Pediatric Emergency Medicine Kawasaki Disease Research Group ,CHARMS Study Group ,Health services and systems - Abstract
BackgroundMultisystem inflammatory syndrome in children (MIS-C) is a novel disease that was identified during the COVID-19 pandemic and is characterised by systemic inflammation following SARS-CoV-2 infection. Early detection of MIS-C is a challenge given its clinical similarities to Kawasaki disease and other acute febrile childhood illnesses. We aimed to develop and validate an artificial intelligence algorithm that can distinguish among MIS-C, Kawasaki disease, and other similar febrile illnesses and aid in the diagnosis of patients in the emergency department and acute care setting.MethodsIn this retrospective model development and validation study, we developed a deep-learning algorithm called KIDMATCH (Kawasaki Disease vs Multisystem Inflammatory Syndrome in Children) using patient age, the five classic clinical Kawasaki disease signs, and 17 laboratory measurements. All features were prospectively collected at the time of initial evaluation from patients diagnosed with Kawasaki disease or other febrile illness between Jan 1, 2009, and Dec 31, 2019, at Rady Children's Hospital in San Diego (CA, USA). For patients with MIS-C, the same data were collected from patients between May 7, 2020, and July 20, 2021, at Rady Children's Hospital, Connecticut Children's Medical Center in Hartford (CT, USA), and Children's Hospital Los Angeles (CA, USA). We trained a two-stage model consisting of feedforward neural networks to distinguish between patients with MIS-C and those without and then those with Kawasaki disease and other febrile illnesses. After internally validating the algorithm using stratified tenfold cross-validation, we incorporated a conformal prediction framework to tag patients with erroneous data or distribution shifts. We finally externally validated KIDMATCH on patients with MIS-C enrolled between April 22, 2020, and July 21, 2021, from Boston Children's Hospital (MA, USA), Children's National Hospital (Washington, DC, USA), and the CHARMS Study Group consortium of 14 US hospitals.Findings1517 patients diagnosed at Rady Children's Hospital between Jan 1, 2009, and June 7, 2021, with MIS-C (n=69), Kawasaki disease (n=775), or other febrile illnesses (n=673) were identified for internal validation, with an additional 16 patients with MIS-C included from Connecticut Children's Medical Center and 50 from Children's Hospital Los Angeles between May 7, 2020, and July 20, 2021. KIDMATCH achieved a median area under the receiver operating characteristic curve during internal validation of 98·8% (IQR 98·0-99·3) in the first stage and 96·0% (95·6-97·2) in the second stage. We externally validated KIDMATCH on 175 patients with MIS-C from Boston Children's Hospital (n=50), Children's National Hospital (n=42), and the CHARMS Study Group consortium of 14 US hospitals (n=83). External validation of KIDMATCH on patients with MIS-C correctly classified 76 of 81 patients (94% accuracy, two rejected by conformal prediction) from 14 hospitals in the CHARMS Study Group consortium, 47 of 49 patients (96% accuracy, one rejected by conformal prediction) from Boston Children's Hospital, and 36 of 40 patients (90% accuracy, two rejected by conformal prediction) from Children's National Hospital.InterpretationKIDMATCH has the potential to aid front-line clinicians to distinguish between MIS-C, Kawasaki disease, and other similar febrile illnesses to allow prompt treatment and prevent severe complications.FundingUS Eunice Kennedy Shriver National Institute of Child Health and Human Development, US National Heart, Lung, and Blood Institute, US Patient-Centered Outcomes Research Institute, US National Library of Medicine, the McCance Foundation, and the Gordon and Marilyn Macklin Foundation.
- Published
- 2022