Development and validation of a machine learning ASA-score to identify candidates for comprehensive preoperative screening and risk stratification.

Objective: The ASA physical status (ASA-PS) is determined by an anesthesia provider or surgeon to communicate co-morbidities relevant to perioperative risk. Assigning an ASA-PS is a clinical decision and there is substantial provider-dependent variability. We developed and externally validated a machine learning-derived algorithm to determine ASA-PS (ML-PS) based on data available in the medical record.
Design: Retrospective multicenter hospital registry study.
Setting: University-affiliated hospital networks.
Patients: Patients who received anesthesia at Beth Israel Deaconess Medical Center (Boston, MA, training [n = 361,602] and internal validation cohorts [n = 90,400]) and Montefiore Medical Center (Bronx, NY, external validation cohort [n = 254,412]).
Measurements: The ML-PS was created using a supervised random forest model with 35 preoperatively available variables. Its predictive ability for 30-day mortality, postoperative ICU admission, and adverse discharge were determined by logistic regression.
Main Results: The anesthesiologist ASA-PS and ML-PS were in agreement in 57.2% of the cases (moderate inter-rater agreement). Compared with anesthesiologist rating, ML-PS assigned more patients into extreme ASA-PS (I and IV), (p < 0.01), and less patients in ASA II and III (p < 0.01). ML-PS and anesthesiologist ASA-PS had excellent predictive values for 30-day mortality, and good predictive values for postoperative ICU admission and adverse discharge. Among the 3594 patients who died within 30 days after surgery, net reclassification improvement analysis revealed that using the ML-PS, 1281 (35.6%) patients were reclassified into the higher clinical risk category compared with anesthesiologist rating. However, in a subgroup of multiple co-morbidity patients, anesthesiologist ASA-PS had a better predictive accuracy than ML-PS.
Conclusions: We created and validated a machine learning physical status based on preoperatively available data. The ability to identify patients at high risk early in the preoperative process independent of the provider's decision is a part of the process we use to standardize the stratified preoperative evaluation of patients scheduled for ambulatory surgery.
Competing Interests: Declaration of Competing Interest All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article or revising it critically for important intellectual content; and (c) approval of the final version. This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue. The authors have no affiliation with any organization with a direct or indirect financial interest in the subject matter discussed in the manuscript. The following authors have affiliations with organizations with direct or indirect financial interest in the subject matter discussed in the manuscript: Matthias Eikermann received unrestricted funds from philanthropic donors Jeffrey and Judith Buzen
(Copyright © 2023 Elsevier Inc. All rights reserved.)