Search

Objectives Social support (SS) and social isolation (SI) are social determinants of health (SDOH) associated with psychiatric outcomes. In electronic health records (EHRs), individual-level SS/SI is typically documented in narrative clinical notes rather than as structured coded data. Natural language processing (NLP) algorithms can automate the otherwise labor-intensive process of extraction of such information. Materials and Methods Psychiatric encounter notes from Mount Sinai Health System (MSHS, n  = 300) and Weill Cornell Medicine (WCM, n  = 225) were annotated to create a gold-standard corpus. A rule-based system (RBS) involving lexicons and a large language model (LLM) using FLAN-T5-XL were developed to identify mentions of SS and SI and their subcategories (eg, social network , instrumental support , and loneliness). Results For extracting SS/SI, the RBS obtained higher macroaveraged F1-scores than the LLM at both MSHS (0.89 versus 0.65) and WCM (0.85 versus 0.82). For extracting the subcategories, the RBS also outperformed the LLM at both MSHS (0.90 versus 0.62) and WCM (0.82 versus 0.81). Discussion and Conclusion Unexpectedly, the RBS outperformed the LLMs across all metrics. An intensive review demonstrates that this finding is due to the divergent approach taken by the RBS and LLM. The RBS was designed and refined to follow the same specific rules as the gold-standard annotations. Conversely, the LLM was more inclusive with categorization and conformed to common English-language understanding. Both approaches offer advantages, although additional replication studies are warranted. [ABSTRACT FROM AUTHOR]

Introduction: Electronic health record (EHR)‐driven phenotyping is a critical first step in generating biomedical knowledge from EHR data. Despite recent progress, current phenotyping approaches are manual, time‐consuming, error‐prone, and platform‐specific. This results in duplication of effort and highly variable results across systems and institutions, and is not scalable or portable. In this work, we investigate how the nascent Clinical Quality Language (CQL) can address these issues and enable high‐throughput, cross‐platform phenotyping. Methods: We selected a clinically validated heart failure (HF) phenotype definition and translated it into CQL, then developed a CQL execution engine to integrate with the Observational Health Data Sciences and Informatics (OHDSI) platform. We executed the phenotype definition at two large academic medical centers, Northwestern Medicine and Weill Cornell Medicine, and conducted results verification (n = 100) to determine precision and recall. We additionally executed the same phenotype definition against two different data platforms, OHDSI and Fast Healthcare Interoperability Resources (FHIR), using the same underlying dataset and compared the results. Results: CQL is expressive enough to represent the HF phenotype definition, including Boolean and aggregate operators, and temporal relationships between data elements. The language design also enabled the implementation of a custom execution engine with relative ease, and results verification at both sites revealed that precision and recall were both 100%. Cross‐platform execution resulted in identical patient cohorts generated by both data platforms. Conclusions: CQL supports the representation of arbitrarily complex phenotype definitions, and our execution engine implementation demonstrated cross‐platform execution against two widely used clinical data platforms. The language thus has the potential to help address current limitations with portability in EHR‐driven phenotyping and scale in learning health systems. [ABSTRACT FROM AUTHOR]

Importance: Diagnostic codes in the Electronic Health Record (EHR) are known to be limited in reporting patient suicidality, and especially in differentiating the levels of suicide severity., Objective: The authors developed and validated a portable natural language processing (NLP) algorithm for detection of suicidal ideation (SI) and suicide-related behavior and attempts (SB/SA) in EHR data. The algorithm was then deployed, and SI and SB/SA ascertainment was compared to that of International Statistical Classification of Diseases (ICD-9 and 10) diagnostic codes., Design: A group of experts designed the pipeline to detect and distinguish suicide severity based on the Columbia-Suicide Severity Rating Scale (C-SSRS). Notes were manually annotated to create the "Gold Standard" with which the algorithm output was evaluated for accuracy., Setting: The algorithm was developed at two academic medical centers, Weill Cornell Medicine (WCM), the Mount Sinai Health System (MSHS), and tested at these two, plus a third, the University of Utah Healthcare Center (UUHSC)., Participants: Notes were from participants with psychiatric encounters at the three institutions., Main Outcomes: The two main outcomes were the accuracy scores of the NLP pipeline and comparison of ascertainment rates to ICD codes., Results: F1 accuracy scores ranged from 0.86-0.97 at the three sites. The NLP rate of detection of SB/SA was almost 30 times higher, and SI was almost 10 times higher, when compared with that of diagnostic codes. NLP detected almost all cases detected by diagnostic codes. No bias in performance was found for race/ethnicity and performance was comparable in psychiatric and non-psychiatric EHRs., Conclusions and Relevance: EHRs from cohorts with psychiatric diagnoses or encounters at WCM, MSHS, and UUHSC had SI and SB/SA extracted using an NLP algorithm based on parameters defined by the C-SSRS. Validity was determined by comparing the algorithm output to manual annotations of clinical notes by domain experts. NLP-detection of SI and SB/SA was compared with that of ICD codes across a range of demographic groups. Algorithm performance was also examined for bias in minoritized groups and in non-psychiatric notes., Key Points: Question: Can we automate the extraction of data available in clinical notes to accurately detect and distinguish patients with suicidal ideation (SI) and suicidal behavior (SB)? Findings: Our Natural Language Processing (NLP) approach was able to identify and distinguish SI and SB at three different hospital systems with benchmarked accuracy scores (above 0.85). The rate of detecting SI and SB using the algorithm was 10-30 times that of diagnostic codes found in the Electronic Health Record. Meaning: Our algorithm renders the use of International Classification of Disease (ICD) diagnostic codes for SI and SB ascertainment obsolete.

Social and behavioral determinants of health (SBDoH) have important roles in shaping people's health. In clinical research studies, especially comparative effectiveness studies, failure to adjust for SBDoH factors will potentially cause confounding issues and misclassification errors in either statistical analyses and machine learning-based models. However, there are limited studies to examine SBDoH factors in clinical outcomes due to the lack of structured SBDoH information in current electronic health record (EHR) systems, while much of the SBDoH information is documented in clinical narratives. Natural language processing (NLP) is thus the key technology to extract such information from unstructured clinical text. However, there is not a mature clinical NLP system focusing on SBDoH. In this study, we examined two state-of-the-art transformer-based NLP models, including BERT and RoBERTa, to extract SBDoH concepts from clinical narratives, applied the best performing model to extract SBDoH concepts on a lung cancer screening patient cohort, and examined the difference of SBDoH information between NLP extracted results and structured EHRs (SBDoH information captured in standard vocabularies such as the International Classification of Diseases codes). The experimental results show that the BERT-based NLP model achieved the best strict/lenient F1-score of 0.8791 and 0.8999, respectively. The comparison between NLP extracted SBDoH information and structured EHRs in the lung cancer patient cohort of 864 patients with 161,933 various types of clinical notes showed that much more detailed information about smoking, education, and employment were only captured in clinical narratives and that it is necessary to use both clinical narratives and structured EHRs to construct a more complete picture of patients' SBDoH factors., (©2021 AMIA - All rights reserved.)

Lack of standardized representation of natural language processing (NLP) components in phenotyping algorithms hinders portability of the phenotyping algorithms and their execution in a high-throughput and reproducible manner. The objective of the study is to develop and evaluate a standard-driven approach - CQL4NLP - that integrates a collection of NLP extensions represented in the HL7 Fast Healthcare Interoperability Resources (FHIR) standard into the clinical quality language (CQL). A minimal NLP data model with 11 NLP-specific data elements was created, including six FHIR NLP extensions. All 11 data elements were identified from their usage in real-world phenotyping algorithms. An NLP ruleset generation mechanism was integrated into the NLP2FHIR pipeline and the NLP rulesets enabled comparable performance for a case study with the identification of obesity comorbidities. The NLP ruleset generation mechanism created a reproducible process for defining the NLP components of a phenotyping algorithm and its execution., (©2021 AMIA - All rights reserved.)

With the increased adoption of electronic health records, data collected for routine clinical care is used for health outcomes and population sciences research, including the identification of phenotypes. In recent years, research networks, such as eMERGE, OHDSI and PCORnet, have been able to increase statistical power and population diversity by combining patient cohorts. These networks share phenotype algorithms that are executed at each participating site. Here we observe experiences with phenotype algorithm portability across seven research networks and propose a generalizable framework for phenotype algorithm portability. Several strategies exist to increase the portability of phenotype algorithms, reducing the implementation effort needed by each site. These include using a common data model, standardized representation of the phenotype algorithm logic, and technical solutions to facilitate federated execution of queries. Portability is achieved by tradeoffs across three domains: Data, Authoring and Implementation, and multiple approaches were observed in representing portable phenotype algorithms. Our proposed framework will help guide future research in operationalizing phenotype algorithm portability at scale., (©2019 AMIA - All rights reserved.)

While natural language processing (NLP) of unstructured clinical narratives holds the potential for patient care and clinical research, portability of NLP approaches across multiple sites remains a major challenge. This study investigated the portability of an NLP system developed initially at the Department of Veterans Affairs (VA) to extract 27 key cardiac concepts from free-text or semi-structured echocardiograms from three academic edical centers: Weill Cornell Medicine, Mayo Clinic and Northwestern Medicine. While the NLP system showed high precision and recall easurements for four target concepts (aortic valve regurgitation, left atrium size at end systole, mitral valve regurgitation, tricuspid valve regurgitation) across all sites, we found moderate or poor results for the remaining concepts and the NLP system performance varied between individual sites., (©2019 AMIA - All rights reserved.)

Acute Kidney Injury (AKI) is the most common cause of organ dysfunction in critically ill adults and prior studies have shown AKI is associated with a significant increase of the mortality risk. Early prediction of the mortality risk for AKI patients can help clinical decision makers better understand the patient condition in time and take appropriate actions. However, AKI is a heterogeneous disease and its cause is complex, which makes such predictions a challenging task. In this paper, we investigate machine learning models for predicting the mortality risk of AKI patients who are stratified according to their AKI stages. With this setup we demonstrate the stratified mortality prediction performance of patients with AKI is better than the results obtained on the mixed population.

The Patient Health Questionnaire-9 (PHQ-9) is a validated instrument for assessing depression severity. While some electronic health record (EHR) systems capture PHQ-9 scores in a structured format, unstructured clinical notes remain the only source in many settings, which presents data retrieval challenges for research and clinical decision support. To address this gap, we extended the open-source Leo natural language processing (NLP) platform to extract PHQ-9 scores from clinical notes and evaluated performance using EHR data for n=123,703 patients who were prescribed antidepressants. Compared to a reference standard, the NLP method exhibited high accuracy (97%), sensitivity (98%), precision (97%), and F-score (97%). Furthermore, of patients with PHQ-9 scores identified by the NLP method, 31% (n=498) had at least one PHQ-9 score clinically indicative of major depressive disorder (MDD), but lacked a structured ICD-9/10 diagnosis code for MDD. This NLP technique may facilitate accurate identification and stratification of patients with depression.

Natural Language Processing (NLP) holds potential for patient care and clinical research, but a gap exists between promise and reality. While some studies have demonstrated portability of NLP systems across multiple sites, challenges remain. Strategies to mitigate these challenges can strive for complex NLP problems using advanced methods (hard-to-reach fruit), or focus on simple NLP problems using practical methods (low-hanging fruit). This paper investigates a practical strategy for NLP portability using extraction of left ventricular ejection fraction (LVEF) as a use case. We used a tool developed at the Department of Veterans Affair (VA) to extract the LVEF values from free-text echocardiograms in the MIMIC-III database. The approach showed an accuracy of 98.4%, sensitivity of 99.4%, a positive predictive value of 98.7%, and F-score of 99.0%. This experience, in which a simple NLP solution proved highly portable with excellent performance, illustrates the point that simple NLP applications may be easier to disseminate and adapt, and in the short term may prove more useful, than complex applications.