Search

BackgroundThe adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited. ObjectiveThis study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics. MethodsWe used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations. ResultsThe retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations. ConclusionsCaution must be taken when interpreting fairness measures’ face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Objective To predict the risk of hospital-acquired pressure injury using machine learning compared with standard care.Design We obtained electronic health records (EHRs) to structure a multilevel cohort of hospitalised patients at risk for pressure injury and then calibrate a machine learning model to predict future pressure injury risk. Optimisation methods combined with multilevel logistic regression were used to develop a predictive algorithm of patient-specific shifts in risk over time. Machine learning methods were tested, including random forests, to identify predictive features for the algorithm. We reported the results of the regression approach as well as the area under the receiver operating characteristics (ROC) curve for predictive models.Setting Hospitalised inpatients.Participants EHRs of 35 001 hospitalisations over 5 years across 2 academic hospitals.Main outcome measure Longitudinal shifts in pressure injury risk.Results The predictive algorithm with features generated by machine learning achieved significantly improved prediction of pressure injury risk (p

Abstract While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians’ autonomy and support them across their entire workflow.

Machine learning (ML) refers to computational algorithms that iteratively improve their ability to recognize patterns in data. The digitization of our healthcare infrastructure is generating an abundance of data from electronic health records, imaging, wearables, and sensors that can be analyzed by ML algorithms to generate personalized risk assessments and promote guideline-directed medical management. ML's strength in generating insights from complex medical data to guide clinical decisions must be balanced with the potential to adversely affect patient privacy, safety, health equity, and clinical interpretability. This review provides a primer on key advances in ML for cardiovascular disease prevention and how they may impact clinical practice.

The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as “endpoints.” A “digital endpoint” is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient’s free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone’s microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer’s disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug’s effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient’s experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.

Tumor mutational burden (TMB), a surrogate for tumor neoepitope burden, is used as a pan-tumor biomarker to identify patients who may benefit from anti-program cell death 1 (PD1) immunotherapy, but it is an imperfect biomarker. Multiple additional genomic characteristics are associated with anti-PD1 responses, but the combined predictive value of these features and the added informativeness of each respective feature remains unknown. We evaluated whether machine learning (ML) approaches using proposed determinants of anti-PD1 response derived from whole exome sequencing (WES) could improve prediction of anti-PD1 responders over TMB alone. Random forest classifiers were trained on publicly available anti-PD1 data (n = 104), and subsequently tested on an independent anti-PD1 cohort (n = 69). Both the training and test datasets included a range of cancer types such as non-small cell lung cancer (NSCLC), head and neck squamous cell carcinoma (HNSCC), melanoma, and smaller numbers of patients from other tumor types. Features used include summaries such as TMB and number of frameshift mutations, as well as more gene-level features such as counts of mutations associated with immune checkpoint response and resistance. Both ML algorithms demonstrated area under the receiver-operator curves (AUC) that exceeded TMB alone (AUC 0.63 “human-guided,” 0.64 “cluster,” and 0.58 TMB alone). Mutations within oncogenes disproportionately modulate anti-PD1 responses relative to their overall contribution to tumor neoepitope burden. The use of a ML algorithm evaluating multiple proposed genomic determinants of anti-PD1 responses modestly improves performance over TMB alone, highlighting the need to integrate other biomarkers to further improve model performance.

OBJECTIVES:. To evaluate factors predictive of clinical progression among coronavirus disease 2019 patients following admission, and whether continuous, automated assessments of patient status may contribute to optimal monitoring and management. DESIGN:. Retrospective cohort for algorithm training, testing, and validation. SETTING:. Eight hospitals across two geographically distinct regions. PATIENTS:. Two-thousand fifteen hospitalized coronavirus disease 2019–positive patients. INTERVENTIONS:. None. MEASUREMENTS AND MAIN RESULTS:. Anticipating Respiratory failure in Coronavirus disease (ARC), a clinically interpretable, continuously monitoring prognostic model of acute respiratory failure in hospitalized coronavirus disease 2019 patients, was developed and validated. An analysis of the most important clinical predictors aligns with key risk factors identified by other investigators but contributes new insights regarding the time at which key factors first begin to exhibit aberrency and distinguishes features predictive of acute respiratory failure in coronavirus disease 2019 versus pneumonia caused by other types of infection. Departing from prior work, ARC was designed to update continuously over time as new observations (vitals and laboratory test results) are recorded in the electronic health record. Validation against data from two geographically distinct health systems showed that the proposed model achieved 75% specificity and 77% sensitivity and predicted acute respiratory failure at a median time of 32 hours prior to onset. Over 80% of true-positive alerts occurred in non-ICU settings. CONCLUSIONS:. Patients admitted to non-ICU environments with coronavirus disease 2019 are at ongoing risk of clinical progression to severe disease, yet it is challenging to anticipate which patients will develop acute respiratory failure. A continuously monitoring prognostic model has potential to facilitate anticipatory rather than reactive approaches to escalation of care (e.g., earlier initiation of treatments for severe disease or structured monitoring and therapeutic interventions for high-risk patients).

Abstract Background This study estimated the extent and predictors of primary nonadherence (i.e., prescriptions made by physicians but not initiated by patients) to methotrexate and to biologics or tofacitinib in rheumatoid arthritis (RA) patients who were newly prescribed these medications. Methods Using administrative claims linked with electronic health records (EHRs) from multiple healthcare provider organizations in the USA, RA patients who received a new prescription for methotrexate or biologics/tofacitinib were identified from EHRs. Claims data were used to ascertain filling or administration status. A logistic regression model for predicting primary nonadherence was developed and tested in training and test samples. Predictors were selected based on clinical judgment and LASSO logistic regression. Results A total of 36.8% of patients newly prescribed methotrexate failed to initiate methotrexate within 2 months; 40.6% of patients newly prescribed biologics/tofacitinib failed to initiate within 3 months. Factors associated with methotrexate primary nonadherence included age, race, region, body mass index, count of active drug ingredients, and certain previously diagnosed and treated conditions at baseline. Factors associated with biologics/tofacitinib primary nonadherence included age, insurance, and certain previously treated conditions at baseline. The area under the receiver operating characteristic curve of the logistic regression model estimated in the training sample and applied to the independent test sample was 0.86 and 0.78 for predicting primary nonadherence to methotrexate and to biologics/tofacitinib, respectively. Conclusions This study confirmed that failure to initiate new prescriptions for methotrexate and biologics/tofacitinib was common in RA patients. It is feasible to predict patients at high risk of primary nonadherence to methotrexate and to biologics/tofacitinib and to target such patients for early interventions to promote adherence.

Machine Learning Special Issue Guest Editors Suchi Saria, Atul Butte, and Aziz Sheikh cut through the hyperbole with an accessible and accurate portrayal of the forefront of machine learning in clinical translation.

Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO2)) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates 2, and, for neonates ≥32 weeks’ gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.

Medication nonadherence costs $300 billion annually in the US. Medicare Advantage plans have a financial incentive to increase medication adherence among members because the Centers for Medicare and Medicaid Services (CMS) now awards substantive bonus payments to such plans, based in part on population adherence to chronic medications. We sought to build an individualized surveillance model that detects early which beneficiaries will fall below the CMS adherence threshold.This was a retrospective study of over 210,000 beneficiaries initiating statins, in a database of private insurance claims, from 2008-2011. A logistic regression model was constructed to use statin adherence from initiation to day 90 to predict beneficiaries who would not meet the CMS measure of proportion of days covered 0.8 or above, from day 91 to 365. The model controlled for 15 additional characteristics. In a sensitivity analysis, we varied the number of days of adherence data used for prediction.Lower adherence in the first 90 days was the strongest predictor of one-year nonadherence, with an odds ratio of 25.0 (95% confidence interval 23.7-26.5) for poor adherence at one year. The model had an area under the receiver operating characteristic curve of 0.80. Sensitivity analysis revealed that predictions of comparable accuracy could be made only 40 days after statin initiation. When members with 30-day supplies for their first statin fill had predictions made at 40 days, and members with 90-day supplies for their first fill had predictions made at 100 days, poor adherence could be predicted with 86% positive predictive value.To preserve their Medicare Star ratings, plan managers should identify or develop effective programs to improve adherence. An individualized surveillance approach can be used to target members who would most benefit, recognizing the tradeoff between improved model performance over time and the advantage of earlier detection.