Search

Background: Evidence-based medicine (EBM) is fundamental to modern clinical practice, requiring clinicians to continually update their knowledge and apply the best clinical evidence in patient care. The practice of EBM faces challenges due to rapid advancements in medical research, leading to information overload for clinicians. The integration of artificial intelligence (AI), specifically Generative Large Language Models (LLMs), offers a promising solution towards managing this complexity. Methods: This study involved the curation of real-world clinical cases across various specialties, converting them into .json files for analysis. LLMs, including proprietary models like ChatGPT 3.5 and 4, Gemini Pro, and open-source models like LLaMA v2 and Mixtral-8x7B, were employed. These models were equipped with tools to retrieve information from case files and make clinical decisions similar to how clinicians must operate in the real world. Model performance was evaluated based on correctness of final answer, judicious use of tools, conformity to guidelines, and resistance to hallucinations. Results: GPT-4 was most capable of autonomous operation in a clinical setting, being generally more effective in ordering relevant investigations and conforming to clinical guidelines. Limitations were observed in terms of model ability to handle complex guidelines and diagnostic nuances. Retrieval Augmented Generation made recommendations more tailored to patients and healthcare systems. Conclusions: LLMs can be made to function as autonomous practitioners of evidence-based medicine. Their ability to utilize tooling can be harnessed to interact with the infrastructure of a real-world healthcare system and perform the tasks of patient management in a guideline directed manner. Prompt engineering may help to further enhance this potential and transform healthcare for the clinician and the patient., Comment: Word count: 4548 words, Figures: 4, Tables: 4

In psychiatry, negative patient descriptions and stigmatizing language can contribute to healthcare disparities in two ways: (1) read by patients they can harm their trust and engagement with the medical center; (2) read by future providers they may negatively influence the future perspective of a patient. By leveraging large language models, this work aims to identify the sentiment expressed in psychiatric clinical notes based on the reader's point of view. Extracting sentences from the Mount Sinai Health System's large and diverse clinical notes, we used prompts and in-context learning to adapt three large language models (GPT-3.5, Llama 2, Mistral) to classify the sentiment conveyed by the sentences according to the provider or non-provider point of view. Results showed that GPT-3.5 aligns best to provider point of view, whereas Mistral aligns best to non-provider point of view., Comment: Oral presentation at NAACL 2024 Queer in AI Workshop

Background: Social support (SS) and social isolation (SI) are social determinants of health (SDOH) associated with psychiatric outcomes. In electronic health records (EHRs), individual-level SS/SI is typically documented as narrative clinical notes rather than structured coded data. Natural language processing (NLP) algorithms can automate the otherwise labor-intensive process of data extraction. Data and Methods: Psychiatric encounter notes from Mount Sinai Health System (MSHS, n=300) and Weill Cornell Medicine (WCM, n=225) were annotated and established a gold standard corpus. A rule-based system (RBS) involving lexicons and a large language model (LLM) using FLAN-T5-XL were developed to identify mentions of SS and SI and their subcategories (e.g., social network, instrumental support, and loneliness). Results: For extracting SS/SI, the RBS obtained higher macro-averaged f-scores than the LLM at both MSHS (0.89 vs. 0.65) and WCM (0.85 vs. 0.82). For extracting subcategories, the RBS also outperformed the LLM at both MSHS (0.90 vs. 0.62) and WCM (0.82 vs. 0.81). Discussion and Conclusion: Unexpectedly, the RBS outperformed the LLMs across all metrics. Intensive review demonstrates that this finding is due to the divergent approach taken by the RBS and LLM. The RBS were designed and refined to follow the same specific rules as the gold standard annotations. Conversely, the LLM were more inclusive with categorization and conformed to common English-language understanding. Both approaches offer advantages and are made available open-source for future testing., Comment: 2 figures, 3 tables

Despite being a unique source of information on patients' status and disease progression, clinical notes are characterized by high levels of duplication and information redundancy. In general domain text, it has been shown that deduplication does not harm language model (LM) pretraining, thus helping reduce the training cost. Although large LMs have proven to learn medical knowledge, they still require specialized domain adaptation for improved downstream clinical tasks. By leveraging large real-world clinical corpora, we first provided a fine-grained characterization of duplicates stemming from common writing practices and clinical relevancy. Second, we demonstrated that deduplicating clinical text can help clinical LMs encode less redundant information in a more efficient manner and do not harm classification tasks via prompt-based learning.

Clinical note classification is a common clinical NLP task. However, annotated data-sets are scarse. Prompt-based learning has recently emerged as an effective method to adapt pre-trained models for text classification using only few training examples. A critical component of prompt design is the definition of the template (i.e. prompt text). The effect of template position, however, has been insufficiently investigated. This seems particularly important in the clinical setting, where task-relevant information is usually sparse in clinical notes. In this study we develop a keyword-optimized template insertion method (KOTI) and show how optimizing position can improve performance on several clinical tasks in a zero-shot and few-shot training setting.

Early detection and intervention are believed to be key to facilitating better outcomes in children with autism, yet the impact of age at treatment start on the outcome is poorly understood. While clinical traits such as language ability have been shown to predict treatment outcome, whether or not and how information at the genomic level can predict treatment outcome is unknown. Leveraging a cohort of toddlers with autism who all received the same standardized intervention at a very young age and provided a blood sample, here we find that very early treatment engagement (i.e., <24 months) leads to greater gains while controlling for time in treatment. Pre-treatment clinical behavioral measures predict 21% of the variance in the rate of skill growth during early intervention. Pre-treatment blood leukocyte gene expression patterns also predict the rate of skill growth, accounting for 13% of the variance in treatment slopes. Results indicated that 295 genes can be prioritized as driving this effect. These treatment-relevant genes highly interact at the protein level, are enriched for differentially histone acetylated genes in autism postmortem cortical tissue, and are normatively highly expressed in a variety of subcortical and cortical areas important for social communication and language development. This work suggests that pre-treatment biological and clinical behavioral characteristics are important for predicting developmental change in the context of early intervention and that individualized pre-treatment biology related to histone acetylation may be key.

Early detection and intervention are believed to be key to facilitating better outcomes in children with autism, yet the impact of age at treatment start on the outcome is poorly understood. While clinical traits such as language ability have been shown to predict treatment outcome, whether or not and how information at the genomic level can predict treatment outcome is unknown. Leveraging a cohort of toddlers with autism who all received the same standardized intervention at a very young age and provided a blood sample, here we find that very early treatment engagement (i.e., <24 months) leads to greater gains while controlling for time in treatment. Pre-treatment clinical behavioral measures predict 21% of the variance in the rate of skill growth during early intervention. Pre-treatment blood leukocyte gene expression patterns also predict the rate of skill growth, accounting for 13% of the variance in treatment slopes. Results indicated that 295 genes can be prioritized as driving this effect. These treatment-relevant genes highly interact at the protein level, are enriched for differentially histone acetylated genes in autism postmortem cortical tissue, and are normatively highly expressed in a variety of subcortical and cortical areas important for social communication and language development. This work suggests that pre-treatment biological and clinical behavioral characteristics are important for predicting developmental change in the context of early intervention and that individualized pre-treatment biology related to histone acetylation may be key.

Contains fulltext : 233963.pdf (Publisher’s version ) (Open Access)

Early detection and intervention are believed to be key to facilitating better outcomes in children with autism, yet the impact of age at treatment start on the outcome is poorly understood. While clinical traits such as language ability have been shown to predict treatment outcome, whether or not and how information at the genomic level can predict treatment outcome is unknown. Leveraging a cohort of toddlers with autism who all received the same standardized intervention at a very young age and provided a blood sample, here we find that very early treatment engagement (i.e., <24 months) leads to greater gains while controlling for time in treatment. Pre-treatment clinical behavioral measures predict 21% of the variance in the rate of skill growth during early intervention. Pre-treatment blood leukocyte gene expression patterns also predict the rate of skill growth, accounting for 13% of the variance in treatment slopes. Results indicated that 295 genes can be prioritized as driving this effect. These treatment-relevant genes highly interact at the protein level, are enriched for differentially histone acetylated genes in autism postmortem cortical tissue, and are normatively highly expressed in a variety of subcortical and cortical areas important for social communication and language development. This work suggests that pre-treatment biological and clinical behavioral characteristics are important for predicting developmental change in the context of early intervention and that individualized pre-treatment biology related to histone acetylation may be key.

Contains fulltext : 233963.pdf (Publisher’s version ) (Open Access)

Contains fulltext : 233963.pdf (Publisher’s version ) (Open Access)

Social-communication (SC) and restricted repetitive behaviors (RRB) are autism diagnostic symptom domains. SC and RRB severity can markedly differ within and between individuals and may be underpinned by different neural circuitry and genetic mechanisms. Modeling SC-RRB balance could help identify how neural circuitry and genetic mechanisms map onto such phenotypic heterogeneity. Here, we developed a phenotypic stratification model that makes highly accurate (97-99%) out-of-sample SC = RRB, SC > RRB, and RRB > SC subtype predictions. Applying this model to resting state fMRI data from the EU-AIMS LEAP dataset (n = 509), we find that while the phenotypic subtypes share many commonalities in terms of intrinsic functional connectivity, they also show replicable differences within some networks compared to a typically-developing group (TD). Specifically, the somatomotor network is hypoconnected with perisylvian circuitry in SC > RRB and visual association circuitry in SC = RRB. The SC = RRB subtype show hyperconnectivity between medial motor and anterior salience circuitry. Genes that are highly expressed within these networks show a differential enrichment pattern with known autism-associated genes, indicating that such circuits are affected by differing autism-associated genomic mechanisms. These results suggest that SC-RRB imbalance subtypes share many commonalities, but also express subtle differences in functional neural circuitry and the genomic underpinnings behind such circuitry.

The recognition of emotional body movement (BM) is impaired in individuals with Autistic Spectrum Disorder ASD, yet it is not clear whether the difficulty is related to the encoding of body motion, emotions, or both. Besides, BM recognition has been traditionally studied using point-light displays stimuli (PLDs) and is still underexplored in individuals with ASD and intellectual disability (ID). In the present study, we investigated the recognition of happy, fearful, and neutral BM in children with ASD with and without ID. In a non-verbal recognition task, participants were asked to recognize pure-body-motion and visible-body-form stimuli (by means of point-light displays-PLDs and full-light displays-FLDs, respectively). We found that the children with ASD were less accurate than TD children in recognizing both the emotional and neutral BM, either when presented as FLDs or PLDs. These results suggest that the difficulty in understanding the observed BM may rely on atypical processing of BM information rather than emotion. Moreover, we found that the accuracy improved with age and IQ only in children with ASD without ID, suggesting that high level of cognitive resources can mediate the acquisition of compensatory mechanisms which develop with age.

The recognition of emotional body movement (BM) is impaired in individuals with Autistic Spectrum Disorder ASD, yet it is not clear whether the difficulty is related to the encoding of body motion, emotions, or both. Besides, BM recognition has been traditionally studied using point-light displays stimuli (PLDs) and is still underexplored in individuals with ASD and intellectual disability (ID). In the present study, we investigated the recognition of happy, fearful, and neutral BM in children with ASD with and without ID. In a non-verbal recognition task, participants were asked to recognize pure-body-motion and visible-body-form stimuli (by means of point-light displays-PLDs and full-light displays-FLDs, respectively). We found that the children with ASD were less accurate than TD children in recognizing both the emotional and neutral BM, either when presented as FLDs or PLDs. These results suggest that the difficulty in understanding the observed BM may rely on atypical processing of BM information rather than emotion. Moreover, we found that the accuracy improved with age and IQ only in children with ASD without ID, suggesting that high level of cognitive resources can mediate the acquisition of compensatory mechanisms which develop with age.

Deriving disease subtypes from electronic health records (EHRs) can guide next-generation personalized medicine. However, challenges in summarizing and representing patient data prevent widespread practice of scalable EHR-based stratification analysis. Here we present an unsupervised framework based on deep learning to process heterogeneous EHRs and derive patient representations that can efficiently and effectively enable patient stratification at scale. We considered EHRs of 1,608,741 patients from a diverse hospital cohort comprising of a total of 57,464 clinical concepts. We introduce a representation learning model based on word embeddings, convolutional neural networks, and autoencoders (i.e., ConvAE) to transform patient trajectories into low-dimensional latent vectors. We evaluated these representations as broadly enabling patient stratification by applying hierarchical clustering to different multi-disease and disease-specific patient cohorts. ConvAE significantly outperformed several baselines in a clustering task to identify patients with different complex conditions, with 2.61 entropy and 0.31 purity average scores. When applied to stratify patients within a certain condition, ConvAE led to various clinically relevant subtypes for different disorders, including type 2 diabetes, Parkinson's disease and Alzheimer's disease, largely related to comorbidities, disease progression, and symptom severity. With these results, we demonstrate that ConvAE can generate patient representations that lead to clinically meaningful insights. This scalable framework can help better understand varying etiologies in heterogeneous sub-populations and unlock patterns for EHR-based research in the realm of personalized medicine., Comment: C.F. and R.M. share senior authorship

Determining the best partition for a dataset can be a challenging task because of 1) the lack of a priori information within an unsupervised learning framework; and 2) the absence of a unique clustering validation approach to evaluate clustering solutions. Here we present reval: a Python package that leverages stability-based relative clustering validation methods to determine best clustering solutions as the ones that best generalize to unseen data. Statistical software, both in R and Python, usually rely on internal validation metrics, such as silhouette, to select the number of clusters that best fits the data. Meanwhile, open-source software solutions that easily implement relative clustering techniques are lacking. Internal validation methods exploit characteristics of the data itself to produce a result, whereas relative approaches attempt to leverage the unknown underlying distribution of data points looking for generalizable and replicable results. The implementation of relative validation methods can further the theory of clustering by enriching the already available methods that can be used to investigate clustering results in different situations and for different data distributions. This work aims at contributing to this effort by developing a stability-based method that selects the best clustering solution as the one that replicates, via supervised learning, on unseen subsets of data. The package works with multiple clustering and classification algorithms, hence allowing both the automatization of the labeling process and the assessment of the stability of different clustering mechanisms.