Search

Background: The lack of automated tools for measuring care quality limits the implementation of a national program to assess guideline-directed care in heart failure with reduced ejection fraction (HFrEF)., Objectives: The authors aimed to automate the identification of patients with HFrEF at hospital discharge, an opportunity to evaluate and improve the quality of care., Methods: The authors developed a novel deep-learning language model for identifying patients with HFrEF from discharge summaries of hospitalizations with heart failure at Yale New Haven Hospital during 2015 to 2019. HFrEF was defined by left ventricular ejection fraction <40% on antecedent echocardiography. The authors externally validated the model at Northwestern Medicine, community hospitals of Yale, and the MIMIC-III (Medical Information Mart for Intensive Care III) database., Results: A total of 13,251 notes from 5,392 unique individuals (age 73 ± 14 years, 48% women), including 2,487 patients with HFrEF (46.1%), were used for model development (train/held-out: 70%/30%). The model achieved an area under receiver-operating characteristic curve (AUROC) of 0.97 and area under precision recall curve (AUPRC) of 0.97 in detecting HFrEF on the held-out set. The model had high performance in identifying HFrEF with AUROC = 0.94 and AUPRC = 0.91 on 19,242 notes from Northwestern Medicine, AUROC = 0.95 and AUPRC = 0.96 on 139 manually abstracted notes from Yale community hospitals, and AUROC = 0.91 and AUPRC = 0.92 on 146 manually reviewed notes from MIMIC-III. Model-based predictions of HFrEF corresponded to a net reclassification improvement of 60.2% ± 1.9% compared with diagnosis codes (P < 0.001)., Conclusions: The authors developed a language model that identifies HFrEF from clinical notes with high precision and accuracy, representing a key element in automating quality assessment for individuals with HFrEF., Competing Interests: Funding Support and Author Disclosures This study was supported by research funding awarded to Dr Khera by the Yale School of Medicine and grant support from the National Heart, Lung, and Blood Institute (NHLBI) of the National Institutes of Health (NIH) under the award K23HL153775. Dr Ahmad is a consultant for Sanofi Aventis, Amgen, and Cytokinetics; and has received research funding from Boehringer Ingelheim, AstraZeneca, Cytokinetics, and Relypsa. Dr Nadkarni has consultancy agreements with AstraZeneca, BioVie, GLG Consulting, Pensieve Health, Reata, Renalytix, Siemens Healthineers and Variant Bio; has received research funding from Goldfinch Bio, and Renalytix; has received honoraria from AstraZeneca, BioVie, Lexicon, Daiichi Sankyo, Meanrini Health, and Reata; has patents or royalties with Renalytix; owns equity and stock options in Pensieve Health and Renalytix as a scientific cofounder; owns equity in Verici Dx; has received financial compensation as a scientific board member and advisor to Renalytix; has served on the advisory board of Neurona Health; and has served in an advisory or leadership role for Pensieve Health and Renalytix. Dr Ahmad has received grants from the Agency for Healthcare Research and Quality, NIH/NHLBI, and American Heart Association; and has received personal fees from Teladoc, Livongo, and Pfizer, outside of the submitted work. Dr Krumholz works under contract with the Centers for Medicare and Medicaid Services to support quality measurement programs; was a recipient of a research grant from Johnson and Johnson, through Yale University, to support clinical trial data sharing; was a recipient of a research agreement, through Yale University, from the Shenzhen Center for Health Information for work to advance intelligent disease prevention and health promotion; collaborates with the National Center for Cardiovascular Diseases in Beijing; receives payment from the Arnold and Porter Law Firm for work related to the Sanofi clopidogrel litigation, from the Martin Baughman Law Firm for work related to the Cook Celect IVC filter litigation, and from the Siegfried and Jensen Law Firm for work related to Vioxx litigation; chairs a Cardiac Scientific Advisory Board for UnitedHealth; was a member of the IBM Watson Health Life Sciences Board; is a member of the Advisory Board for Element Science, the Advisory Board for Facebook, and the Physician Advisory Board for Aetna; and is the cofounder of Hugo Health, a personal health information platform, and cofounder of Refactor Health, a health care AI-augmented data management company. Dr Khera is an Associate Editor of JAMA; receives support from the NHLBI of the NIH (under awards R01HL167858 and K23HL153775) and the Doris Duke Charitable Foundation (under award 2022060); has received research support, through Yale, from Bristol Myers Squibb, Novo Nordisk, and BridgeBio; is a coinventor of U.S. Pending Patent Applications 63/562,335, 63/177,117, 63/428,569, 63/346,610, 63/484,426, 63/508,315, and 63/606,203; and is a co-founder of Ensight-AI, Inc and Evidence2Health, health platforms to improve cardiovascular diagnosis and evidence-based cardiovascular care. Dr Nargesi has received funding from the NHLBI under award number T32HL007604. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose., (Copyright © 2025. Published by Elsevier Inc.) more...

Importance: Serial functional status assessments are critical to heart failure (HF) management but are often described narratively in documentation, limiting their use in quality improvement or patient selection for clinical trials., Objective: To develop and validate a deep learning natural language processing (NLP) strategy for extracting functional status assessments from unstructured clinical documentation., Design, Setting, and Participants: This diagnostic study used electronic health record data collected from January 1, 2013, through June 30, 2022, from patients diagnosed with HF seeking outpatient care within 3 large practice networks in Connecticut (Yale New Haven Hospital [YNHH], Northeast Medical Group [NMG], and Greenwich Hospital [GH]). Expert-annotated notes were used for NLP model development and validation. Data were analyzed from February to April 2024., Exposures: Development and validation of NLP models to detect explicit New York Heart Association (NYHA) classification, HF symptoms during activity or rest, and frequency of functional status assessments., Main Outcomes and Measures: Outcomes of interest were model performance metrics, including area under the receiver operating characteristic curve (AUROC), and frequency of NYHA class documentation and HF symptom descriptions in unannotated notes., Results: This study included 34 070 patients with HF (mean [SD] age 76.1 [12.6] years; 17 728 [52.0]% female). Among 3000 expert-annotated notes (2000 from YNHH and 500 each from NMG and GH), 374 notes (12.4%) mentioned NYHA class and 1190 notes (39.7%) described HF symptoms. The NYHA class detection model achieved a class-weighted AUROC of 0.99 (95% CI, 0.98-1.00) at YNHH, the development site. At the 2 validation sites, NMG and GH, the model achieved class-weighted AUROCs of 0.98 (95% CI, 0.96-1.00) and 0.98 (95% CI, 0.92-1.00), respectively. The model for detecting activity- or rest-related symptoms achieved an AUROC of 0.94 (95% CI, 0.89-0.98) at YNHH, 0.94 (95% CI, 0.91-0.97) at NMG, and 0.95 (95% CI, 0.92-0.99) at GH. Deploying the NYHA model among 182 308 unannotated notes from the 3 sites identified 23 830 (13.1%) notes with NYHA mentions, specifically 10 913 notes (6.0%) with class I, 12 034 notes (6.6%) with classes II or III, and 883 notes (0.5%) with class IV. An additional 19 730 encounters (10.8%) could be classified into functional status groups based on activity- or rest-related symptoms, resulting in a total of 43 560 medical notes (23.9%) categorized by NYHA, an 83% increase compared with explicit mentions alone., Conclusions and Relevance: In this diagnostic study of 34 070 patients with HF, the NLP approach accurately extracted a patient's NYHA symptom class and activity- or rest-related HF symptoms from clinical notes, enhancing the ability to track optimal care delivery and identify patients eligible for clinical trial participation from unstructured documentation. more...