Search

PURPOSE Patients undergoing outpatient radiotherapy (RT) or chemoradiation (CRT) frequently require acute care (emergency department evaluation or hospitalization). Machine learning (ML) may guide interventions to reduce this risk. There are limited prospective studies investigating the clinical impact of ML in health care. The objective of this study was to determine whether ML can identify high-risk patients and direct mandatory twice-weekly clinical evaluation to reduce acute care visits during treatment. PATIENTS AND METHODS During this single-institution randomized quality improvement study (ClinicalTrials.gov identifier: NCT04277650 ), 963 outpatient adult courses of RT and CRT started from January 7 to June 30, 2019, were evaluated by an ML algorithm. Among these, 311 courses identified by ML as high risk (> 10% risk of acute care during treatment) were randomized to standard once-weekly clinical evaluation (n = 157) or mandatory twice-weekly evaluation (n = 154). Both arms allowed additional evaluations on the basis of clinician discretion. The primary end point was the rate of acute care visits during RT. Model performance was evaluated using receiver operating characteristic area under the curve (AUC) and decile calibration plots. RESULTS Twice-weekly evaluation reduced rates of acute care during treatment from 22.3% to 12.3% (difference, −10.0%; 95% CI, −18.3 to −1.6; relative risk, 0.556; 95% CI, 0.332 to 0.924; P = .02). Low-risk patients had a 2.7% acute care rate. Model discrimination was good in high- and low-risk patients undergoing standard once-weekly evaluation (AUC, 0.851). CONCLUSION In this prospective randomized study, ML accurately triaged patients undergoing RT and CRT, directing clinical management with reduced acute care rates versus standard of care. This prospective study demonstrates the potential benefit of ML in health care and offers opportunities to enhance care quality and reduce health care costs.

1509 Background: SHIELD-RT was a randomized controlled quality improvement study (NCT03775265) that implemented electronic health record-based machine learning (ML) to direct supplemental visits for high risk (HR) patients undergoing radiotherapy (RT). Acute care visits (ER visits or hospitalizations) were reduced from 22% to 12%. We evaluated the costs associated with acute visits in this study. Methods: Patients who initiated RT between 1/7/19 and 6/30/19 at a single institution were evaluated by a ML algorithm to identify HR courses (>10% risk of acute visit during RT). HR patients were randomized to standard weekly (S) or intervention of twice weekly (TW) evaluation during RT. Cost data associated with acute visits were obtained and compared between patients who underwent S or TW evaluations. Missing cost data were imputed using disease related groups (DRGs). Mean costs (standard deviation) were compared between arms with non-parametric Wilcoxon Rank Sum tests. Results: 311 HR courses were identified and randomized to either S (n=157) or TW (n=154) evaluations during RT. 85 patients (S: 51; TW: 34) had 121 distinct acute care visits (S: 74; TW: 47). Patients in the TW evaluation arm had fewer hospitalizations (29 vs 41) and ER visits (18 vs 33) than those in the S arm. There were fewer acute visits per patient in the TW arm (0.34) compared to S arm (0.49). Actual cost data was available for 102 visits at our institution, and imputed for 19 outside hospital visits. Mean cost associated with acute visits was lower in the TW arm ($1939, SD $5912) compared with the S arm ($4002, SD $11568; p=0.03). Differences in mean cost between arms are presented in the table. Conclusions: ML-directed evaluations for HR patients undergoing RT resulted in decreased costs of ER visits and hospitalizations. Costs were decreased across revenue centers, with the largest difference related to inpatient room costs. Future analyses will incorporate intervention costs, which are currently bundled with RT reimbursement.[Table: see text]