Search

PURPOSE Patients undergoing outpatient radiotherapy (RT) or chemoradiation (CRT) frequently require acute care (emergency department evaluation or hospitalization). Machine learning (ML) may guide interventions to reduce this risk. There are limited prospective studies investigating the clinical impact of ML in health care. The objective of this study was to determine whether ML can identify high-risk patients and direct mandatory twice-weekly clinical evaluation to reduce acute care visits during treatment. PATIENTS AND METHODS During this single-institution randomized quality improvement study (ClinicalTrials.gov identifier: NCT04277650 ), 963 outpatient adult courses of RT and CRT started from January 7 to June 30, 2019, were evaluated by an ML algorithm. Among these, 311 courses identified by ML as high risk (> 10% risk of acute care during treatment) were randomized to standard once-weekly clinical evaluation (n = 157) or mandatory twice-weekly evaluation (n = 154). Both arms allowed additional evaluations on the basis of clinician discretion. The primary end point was the rate of acute care visits during RT. Model performance was evaluated using receiver operating characteristic area under the curve (AUC) and decile calibration plots. RESULTS Twice-weekly evaluation reduced rates of acute care during treatment from 22.3% to 12.3% (difference, −10.0%; 95% CI, −18.3 to −1.6; relative risk, 0.556; 95% CI, 0.332 to 0.924; P = .02). Low-risk patients had a 2.7% acute care rate. Model discrimination was good in high- and low-risk patients undergoing standard once-weekly evaluation (AUC, 0.851). CONCLUSION In this prospective randomized study, ML accurately triaged patients undergoing RT and CRT, directing clinical management with reduced acute care rates versus standard of care. This prospective study demonstrates the potential benefit of ML in health care and offers opportunities to enhance care quality and reduce health care costs.

PURPOSE/OBJECTIVE(S) Mortality prediction is critical to appropriate cancer care planning. This has become a topic of interest, with machine learning (ML) tools demonstrating accurate binary predictions for mortality at specific time points. There are limited prospective data validating the clinical utility of health care ML tools. We previously developed a ML algorithm which predicts risk of acute care (ER visits or hospitalizations). SHIELD-RT was a randomized controlled study which demonstrated that a ML algorithm could direct supplemental clinic visits to reduce the rate of acute care during RT from 22% to 12%. While the algorithm was trained for acute care visits, we sought to determine whether ML risk would facilitate multiple clinical uses through survival prediction in this prospective study. MATERIALS/METHODS Patients who initiated RT from 1/7/2019-6/30/19 at a single institution were evaluated during the first week of treatment by the electronic health record-based ML algorithm. High risk patients (> 10% risk of acute visit) were randomized to standard weekly evaluations (S) or twice weekly evaluations (TW). Patient, disease, and treatment factors were collected prospectively. Survival was recorded retrospectively. Unadjusted OS was estimated with the Kaplan Meier method and compared between arms with the log rank test. Cox proportional hazards models were used to estimate the association of covariates with OS after adjustment for study arm. Hazard ratios (HRs) and 95% confidence intervals (CIs) are reported. RESULTS Patients undergoing 311 distinct courses were randomized to S (n = 157) or TW (n = 154) evaluations. The 1-year OS was similar for those who underwent S (58%, 95% CI 50%-65%) or TW (66%, 95% CI 58%-73%) evaluations, P = 0.35. ML risk on a continuous basis was significantly associated with survival (HR 4.27, 95% CI 0.98-18.6, P = 0.05). Other factors associated with OS included age (HR 1.01, 95% CI 1.00-1.03, P = 0.04), palliative treatment vs curative treatment (HR 6.59 (95% CI 4.62-9.41, < 0.001), widely metastatic disease vs no metastatic disease (HR 6.86, 95% CI 4.74-9.92, P < 0.001). Treatment of certain disease sites were also associated with OS: brain metastases (HR 5.00, 95% CI 3.16-7.93, P < 0.001), bone metastases (HR 4.69, 95% CI 3.09-7.12, P < 0.001), thoracic (HR 2.44, 95% CI 1.61-3.7, P < 0.001), head and neck cancer (HR 0.44, 95% CI 0.22-0.91, P = 0.03), genitourinary cancer (HR 0.32, 95% CI 0.12-0.85, P = 0.02). Admission during RT + 15 days was associated with decreased OS (HR 2.32, 95% CI 1.58-3.40, P < 0.001). CONCLUSION Our ML algorithm, developed to predict acute care, also predicted OS. This supports the correlation between the two event types and demonstrates the utility of ML to provide multiple predictive functions. Implementation of predictive ML algorithms into oncology clinical workflows may aid in prognostication to guide personalized patient care and influence treatment decisions. (NCT03775265).

1509 Background: SHIELD-RT was a randomized controlled quality improvement study (NCT03775265) that implemented electronic health record-based machine learning (ML) to direct supplemental visits for high risk (HR) patients undergoing radiotherapy (RT). Acute care visits (ER visits or hospitalizations) were reduced from 22% to 12%. We evaluated the costs associated with acute visits in this study. Methods: Patients who initiated RT between 1/7/19 and 6/30/19 at a single institution were evaluated by a ML algorithm to identify HR courses (>10% risk of acute visit during RT). HR patients were randomized to standard weekly (S) or intervention of twice weekly (TW) evaluation during RT. Cost data associated with acute visits were obtained and compared between patients who underwent S or TW evaluations. Missing cost data were imputed using disease related groups (DRGs). Mean costs (standard deviation) were compared between arms with non-parametric Wilcoxon Rank Sum tests. Results: 311 HR courses were identified and randomized to either S (n=157) or TW (n=154) evaluations during RT. 85 patients (S: 51; TW: 34) had 121 distinct acute care visits (S: 74; TW: 47). Patients in the TW evaluation arm had fewer hospitalizations (29 vs 41) and ER visits (18 vs 33) than those in the S arm. There were fewer acute visits per patient in the TW arm (0.34) compared to S arm (0.49). Actual cost data was available for 102 visits at our institution, and imputed for 19 outside hospital visits. Mean cost associated with acute visits was lower in the TW arm ($1939, SD $5912) compared with the S arm ($4002, SD $11568; p=0.03). Differences in mean cost between arms are presented in the table. Conclusions: ML-directed evaluations for HR patients undergoing RT resulted in decreased costs of ER visits and hospitalizations. Costs were decreased across revenue centers, with the largest difference related to inpatient room costs. Future analyses will incorporate intervention costs, which are currently bundled with RT reimbursement.[Table: see text]