Search

Background: While many patients seem to recover from SARS-CoV-2 infections, many patients report experiencing SARS-CoV-2 symptoms for weeks or months after their acute COVID-19 ends, even developing new symptoms weeks after infection. These long-term effects are called post-acute sequelae of SARS-CoV-2 (PASC) or, more commonly, Long COVID. The overall prevalence of Long COVID is currently unknown, and tools are needed to help identify patients at risk for developing long COVID., Methods: A working group of the Rapid Acceleration of Diagnostics-radical (RADx-rad) program, comprised of individuals from various NIH institutes and centers, in collaboration with REsearching COVID to Enhance Recovery (RECOVER) developed and organized the Long COVID Computational Challenge (L3C), a community challenge aimed at incentivizing the broader scientific community to develop interpretable and accurate methods for identifying patients at risk of developing Long COVID. From August 2022 to December 2022, participants developed Long COVID risk prediction algorithms using the National COVID Cohort Collaborative (N3C) data enclave, a harmonized data repository from over 75 healthcare institutions from across the United States (U.S.)., Findings: Over the course of the challenge, 74 teams designed and built 35 Long COVID prediction models using the N3C data enclave. The top 10 teams all scored above a 0.80 Area Under the Receiver Operator Curve (AUROC) with the highest scoring model achieving a mean AUROC of 0.895. Included in the top submission was a visualization dashboard that built timelines for each patient, updating the risk of a patient developing Long COVID in response to clinical events., Interpretation: As a result of L3C, federal reviewers identified multiple machine learning models that can be used to identify patients at risk for developing Long COVID. Many of the teams used approaches in their submissions which can be applied to future clinical prediction questions., Funding: Research reported in this RADx® Rad publication was supported by the National Institutes of Health. Timothy Bergquist, Johanna Loomba, and Emily Pfaff were supported by Axle Subcontract: NCATS-STSS-P00438., Competing Interests: Declaration of interests Danielle Mowery serves as an unpaid member of the Epic Cosmos Governing Council. Romain Pirracchio received funding from the FDA CERSI grant U01FD005978 and the PCORI grant P0562155 and received a consulting honorarium from Phillips. Martin van der Laan received funding from the NIAID grant 5R01AI074345. Johanna Loomba received contract funding from the NIH RECOVER program. Emily Pfaff received funding from the NIH and PCORI. The views expressed in this manuscript are solely those of the authors and do not necessarily represent those of the National Institutes of Health, the U.S. Department of Health and Human Services or the U.S. government. Qi Long was supported by grants from the NIH., (Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)

In this work we introduce the personalized online super learner (POSL), an online personalizable ensemble machine learning algorithm for streaming data. POSL optimizes predictions with respect to baseline covariates, so personalization can vary from completely individualized, that is, optimization with respect to subject ID, to many individuals, that is, optimization with respect to common baseline covariates. As an online algorithm, POSL learns in real time. As a super learner, POSL is grounded in statistical optimality theory and can leverage a diversity of candidate algorithms, including online algorithms with different training and update times, fixed/offline algorithms that are not updated during POSL's fitting procedure, pooled algorithms that learn from many individuals' time series, and individualized algorithms that learn from within a single time series. POSL's ensembling of the candidates can depend on the amount of data collected, the stationarity of the time series, and the mutual characteristics of a group of time series. Depending on the underlying data-generating process and the information available in the data, POSL is able to adapt to learning across samples, through time, or both. For a range of simulations that reflect realistic forecasting scenarios and in a medical application, we examine the performance of POSL relative to other current ensembling and online learning methods. We show that POSL is able to provide reliable predictions for both short and long time series, and it's able to adjust to changing data-generating environments. We further cultivate POSL's practicality by extending it to settings where time series dynamically enter and exit., (© 2023 John Wiley & Sons Ltd.)

Background: Deviation from guidelines is frequent in emergency situations, and this may lead to increased mortality. Probably because of time constraints, 55% is the greatest reported guidelines compliance rate in severe trauma patients. This study aimed to identify among all available recommendations a reasonable bundle of items that should be followed to optimize the outcome of hemorrhagic shocks (HSs) and severe traumatic brain injuries (TBIs)., Methods: We first estimated the compliance with French and European guidelines using the data from the French TraumaBase registry. Then, we used a machine learning procedure to reduce the number of recommendations into a minimal set of items to be followed to minimize 7-day mortality. We evaluated the bundles using an external validation cohort., Results: This study included 5,924 trauma patients (1,414 HS and 4,955 TBI) between 2011 and August 2019 and studied compliance to 36 recommendation items. Overall compliance rate to recommendation items was 71.6% and 66.9% for HS and TBI, respectively. In HS, compliance was significantly associated with 7-day decreased mortality in univariate analysis but not in multivariate analysis (risk ratio [RR], 0.91; 95% confidence interval [CI], 0.90-1.17; p = 0.06). In TBI, compliance was significantly associated with decreased mortality in univariate and multivariate analysis (RR, 0.85; 95% CI, 0.75-0.92; p = 0.01). For HS, the bundle included 13 recommendation items. In the validation cohort, when this bundle was applied, patients were found to have a lower 7-day mortality rate (RR, 0.46; 95% CI, 0.27-0.63; p = 0.01). In TBI, the bundle included seven items. In the validation cohort, when this bundle was applied, patients had a lower 7-day mortality rate (RR, 0.55; 95% CI, 0.34-0.71; p = 0.02)., Discussion: Using a machine-learning procedure, we were able to identify a subset of recommendations that minimizes 7-day mortality following traumatic HS and TBI. These two bundles remain to be evaluated in a prospective manner., Level of Evidence: Care Management, level II., (Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.)

Importance: The survival benefit of corticosteroids in septic shock remains uncertain., Objective: To estimate the individual treatment effect (ITE) of corticosteroids in adults with septic shock in intensive care units using machine learning and to evaluate the net benefit of corticosteroids when the decision to treat is based on the individual estimated absolute treatment effect., Design, Setting, and Participants: This cohort study used individual patient data from 4 trials on steroid supplementation in adults with septic shock as a training cohort to model the ITE using an ensemble machine learning approach. Data from a double-blinded, placebo-controlled randomized clinical trial comparing hydrocortisone with placebo were used for external validation. Data analysis was conducted from September 2019 to February 2020., Exposures: Intravenous hydrocortisone 50 mg dose every 6 hours for 5 to 7 days with or without enteral 50 μg of fludrocortisone daily for 7 days. The control was either the placebo or usual care., Main Outcomes and Measures: All-cause 90-day mortality., Results: A total of 2548 participants were included in the development cohort, with median (interquartile range [IQR]) age of 66 (55-76) years and 1656 (65.0%) men. The median (IQR) Simplified Acute Physiology Score (SAPS II) was 55 [42-69], and median (IQR) Sepsis-related Organ Failure Assessment score on day 1 was 11 (9-13). The crude pooled relative risk (RR) of death at 90 days was 0.89 (95% CI, 0.83 to 0.96) in favor of corticosteroids. According to the optimal individual model, the estimated median absolute risk reduction was of 2.90% (95% CI, 2.79% to 3.01%). In the external validation cohort of 75 patients, the area under the curve of the optimal individual model was 0.77 (95% CI, 0.59 to 0.92). For any number willing to treat (NWT; defined as the acceptable number of people to treat to avoid 1 additional outcome considering the risk of harm associated with the treatment) less than 25, the net benefit of treating all patients vs treating nobody was negative. When the NWT was 25, the net benefit was 0.01 for the treat all with hydrocortisone strategy, -0.01 for treat all with hydrocortisone and fludrocortisone strategy, 0.06 for the treat by SAPS II strategy, and 0.31 for the treat by optimal individual model strategy. The net benefit of the SAPS II and the optimal individual model treatment strategies converged to zero for a smaller number willing to treat, but the individual model was consistently superior than model based on the SAPS II score., Conclusions and Relevance: These findings suggest that an individualized treatment strategy to decide which patient with septic shock to treat with corticosteroids yielded positive net benefit regardless of potential corticosteroid-associated side effects.

Background: Massive transfusion protocols to treat postinjury hemorrhage are based on predefined blood product transfusion ratios followed by goal-directed transfusion based on patient's clinical evolution. However, it remains unclear how these transfusion ratios impact patient outcomes over time from injury., Methods: The Pragmatic, Randomized Optimal Platelet and Plasma Ratios (PROPPR) is a phase 3, randomized controlled trial, across 12 Level I trauma centers in North America. From 2012 to 2013, 680 severely injured patients required massive transfusion. We used semiparametric machine learning techniques and causal inference methods to augment the intent-to-treat analysis of PROPPR, estimating the dynamic relationship between transfusion ratios and outcomes: mortality and hemostasis at different timepoints during the first 24 hours after admission., Results: In the intention-to-treat analysis, the 1:1:1 group tended to have decreased mortality, but with no statistical significance. For patients in whom hemostasis took longer than 2 hours, the 1:1:1 ratio was associated with a higher probability of hemostasis, statistically significant from the 4 hour on. In the per-protocol, actual-transfusion-ratios-received analysis, during four successive time intervals, no significant association was found between the actual ratios and mortality. When comparing patient groups who received both high plasma/PRBC and high platelet/PRBC ratios to the group of low ratios in both, the relative risk of achieving hemostasis was 2.49 (95% confidence interval, 1.19-5.22) during the third hour after admission, suggesting a significant beneficial impact of higher transfusion ratios of plasma and platelets on hemostasis., Conclusion: Our results suggest that the impact of transfusion ratios on hemostasis is dynamic. Overall, the transfusion ratios had no significant impact on mortality over time. However, receiving higher ratios of platelets and plasma relative to red blood cells hastens hemostasis in subjects who have yet to achieve hemostasis within 3 hours after hospital admission., Level of Evidence: Therapeutic IV.

Background: Acute aortic syndrome (AAS) comprises a complex and potentially fatal group of conditions requiring emergency specialist management. The aim of this study was to build a prediction algorithm to assist prehospital triage of AAS., Methods: Details of consecutive patients enrolled in a regional specialist aortic network were collected prospectively. Two prediction algorithms for AAS based on logistic regression and an ensemble machine learning method called SuperLearner (SL) were developed. Undertriage was defined as the proportion of patients with AAS not transported to the specialist aortic centre, and overtriage as the proportion of patients with alternative diagnoses but transported to the specialist aortic centre., Results: Data for 976 hospital admissions between February 2010 and June 2017 were included; 609 (62·4 per cent) had AAS. Overtriage and undertriage rates were 52·3 and 16·1 per cent respectively. The population was divided into a training cohort (743 patients) and a validation cohort (233). The area under the receiver operating characteristic (ROC) curve values for the logistic regression score and the SL were 0·68 (95 per cent c.i. 0·64 to 0·72) and 0·87 (0·84 to 0·89) respectively (P < 0·001) in the training cohort, and 0·67 (0·60 to 0·74) and 0·73 (0·66 to 0·79) in the validation cohort (P = 0·038). The logistic regression score was associated with undertriage and overtriage rates of 33·7 (bootstrapped 95 per cent c.i. 29·3 to 38·3) and 7·2 (4·8 to 9·8) per cent respectively, whereas the SL yielded undertriage and overtriage rates of 1·0 (0·3 to 2·0) and 30·2 (25·8 to 34·8) per cent respectively., Conclusion: A machine learning prediction model performed well in discriminating AAS and could be clinically useful in prehospital triage of patients with suspected AAS., (© 2020 BJS Society Ltd Published by John Wiley & Sons Ltd.)

Background: Acute hypotensive episodes (AHE), defined as a drop in the mean arterial pressure (MAP) <65 mm Hg lasting at least 5 consecutive minutes, are among the most critical events in the intensive care unit (ICU). They are known to be associated with adverse outcome in critically ill patients. AHE prediction is of prime interest because it could allow for treatment adjustment to predict or shorten AHE., Methods: The Super Learner (SL) algorithm is an ensemble machine-learning algorithm that we specifically trained to predict an AHE 10 minutes in advance. Potential predictors included age, sex, type of care unit, severity scores, and time-evolving characteristics such as mechanical ventilation, vasopressors, or sedation medication as well as features extracted from physiological signals: heart rate, pulse oximetry, and arterial blood pressure. The algorithm was trained on the Medical Information Mart for Intensive Care dataset (MIMIC II) database. Internal validation was based on the area under the receiver operating characteristic curve (AUROC) and the Brier score (BS). External validation was performed using an external dataset from Lariboisière hospital, Paris, France., Results: Among 1151 patients included, 826 (72%) patients had at least 1 AHE during their ICU stay. Using 1 single random period per patient, the SL algorithm with Haar wavelets transform preprocessing was associated with an AUROC of 0.929 (95% confidence interval [CI], 0.899-0.958) and a BS of 0.08. Using all available periods for each patient, SL with Haar wavelets transform preprocessing was associated with an AUROC of 0.890 (95% CI, 0.886-0.895) and a BS of 0.11. In the external validation cohort, the AUROC reached 0.884 (95% CI, 0.775-0.993) with 1 random period per patient and 0.889 (0.768-1) with all available periods and BSs <0.1., Conclusions: The SL algorithm exhibits good performance for the prediction of an AHE 10 minutes ahead of time. It allows an efficient, robust, and rapid evaluation of the risk of hypotension that opens the way to routine use.

Machine learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption is limited by the level of trust afforded by given models. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of humans and machines. Here, we present expert-augmented machine learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We used a large dataset of intensive-care patient data to derive 126 decision rules that predict hospital mortality. Using an online platform, we asked 15 clinicians to assess the relative risk of the subpopulation defined by each rule compared to the total sample. We compared the clinician-assessed risk to the empirical risk and found that, while clinicians agreed with the data in most cases, there were notable exceptions where they overestimated or underestimated the true risk. Studying the rules with greatest disagreement, we identified problems with the training data, including one miscoded variable and one hidden confounder. Filtering the rules based on the extent of disagreement between clinician-assessed risk and empirical risk, we improved performance on out-of-sample data and were able to train with less data. EAML provides a platform for automated creation of problem-specific priors, which help build robust and dependable machine-learning models in critical applications., Competing Interests: Competing interest statement: The editor, P.J.B., and one of the authors, M.J.v.d.L., are at the same institution (University of California, Berkeley)., (Copyright © 2020 the Author(s). Published by PNAS.)

Historically, personalised medicine has been synonymous with pharmacogenomics and oncology. We argue for a new framework for personalised medicine analytics that capitalises on more detailed patient-level data and leverages recent advances in causal inference and machine learning tailored towards decision support applicable to critically ill patients. We discuss how advances in data technology and statistics are providing new opportunities for asking more targeted questions regarding patient treatment, and how this can be applied in the intensive care unit to better predict patient-centred outcomes, help in the discovery of new treatment regimens associated with improved outcomes, and ultimately how these rules can be learned in real-time for the patient., (Copyright © 2018 Société française d’anesthésie et de réanimation (Sfar). Published by Elsevier Masson SAS. All rights reserved.)

Consistency of the propensity score estimators rely on correct specification of the propensity score model. The propensity score is frequently estimated using a main effect logistic regression. It has recently been shown that the use of ensemble machine learning algorithms, such as the Super Learner, could improve covariate balance and reduce bias in a meaningful manner in the case of serious model misspecification for treatment assignment. However, the loss functions normally used by the Super Learner may not be appropriate for propensity score estimation since the goal in this problem is not to optimize propensity score prediction but rather to achieve the best possible balance in the covariate distribution between treatment groups. In a simulation study, we evaluated the benefit of a modification of the Super Learner by propensity score estimation geared toward achieving covariate balance between the treated and untreated after matching on the propensity score. Our simulation study included six different scenarios characterized by various degrees of deviation from the usual main term logistic model for the true propensity score and outcome as well as the presence (or not) of instrumental variables. Our results suggest that the use of this adapted Super Learner to estimate the propensity score can further improve the robustness of propensity score matching estimators.

Background: Improved mortality prediction for patients in intensive care units is a big challenge. Many severity scores have been proposed, but findings of validation studies have shown that they are not adequately calibrated. The Super ICU Learner Algorithm (SICULA), an ensemble machine learning technique that uses multiple learning algorithms to obtain better prediction performance, does at least as well as the best member of its library. We aimed to assess whether the Super Learner could provide a new mortality prediction algorithm for patients in intensive care units, and to assess its performance compared with other scoring systems., Methods: From January, 2001, to December, 2008, we used the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) database (version 26) including all patients admitted to an intensive care unit at the Beth Israel Deaconess Medical Centre, Boston, MA, USA. We assessed the calibration, discrimination, and risk classification of predicted hospital mortality based on Super Learner compared with SAPS-II, APACHE-II, and SOFA. We calculated performance measures with cross-validation to avoid making biased assessments. Our proposed score was then externally validated on a dataset of 200 randomly selected patients admitted at the intensive care unit of Hôpital Européen Georges-Pompidou, Paris, France, between Sept 1, 2013, and June, 30, 2014. The primary outcome was hospital mortality. The explanatory variables were the same as those included in the SAPS II score., Findings: 24,508 patients were included, with median SAPS-II of 38 (IQR 27-51) and median SOFA of 5 (IQR 2-8). 3002 of 24,508 (12%) patients died in the Beth Israel Deaconess Medical Centre. We produced two sets of predictions based on the Super Learner; the first based on the 17 variables as they appear in the SAPS-II score (SL1), and the second, on the original, untransformed variables (SL2). The two versions yielded average predicted probabilities of death of 0·12 (IQR 0·02-0·16) and 0·13 (0·01-0·19), whereas the corresponding value for SOFA was 0·12 (0·05-0·15) and for SAPS-II 0·30 (0·08-0·48). The cross-validated area under the receiver operating characteristic curve (AUROC) for SAPS-II was 0·78 (95% CI 0·77-0·78) and 0·71 (0·70-0·72) for SOFA. Super Learner had an AUROC of 0·85 (0·84-0·85) when the explanatory variables were categorised as in SAPS-II, and of 0·88 (0·87-0·89) when the same explanatory variables were included without any transformation. Additionally, Super Learner showed better calibration properties than previous score systems. On the external validation dataset, the AUROC was 0·94 (0·90-0·98) and calibration properties were good., Interpretation: Compared with conventional severity scores, Super Learner offers improved performance for predicting hospital mortality in patients in intensive care units. A user-friendly implementation is available online and should be useful for clinicians seeking to validate our score., Funding: Fulbright Foundation, Assistance Publique-Hôpitaux de Paris, Doris Duke Clinical Scientist Development Award, and the NIH., (Copyright © 2015 Elsevier Ltd. All rights reserved.)