Search

The introduction of Large Language Models (LLMs) has advanced data representation and analysis, bringing significant progress in their use for medical questions and answering. Despite these advancements, integrating tabular data, especially numerical data pivotal in clinical contexts, into LLM paradigms has not been thoroughly explored. In this study, we examine the effectiveness of vector representations from last hidden states of LLMs for medical diagnostics and prognostics using electronic health record (EHR) data. We compare the performance of these embeddings with that of raw numerical EHR data when used as feature inputs to traditional machine learning (ML) algorithms that excel at tabular data learning, such as eXtreme Gradient Boosting. We focus on instruction-tuned LLMs in a zero-shot setting to represent abnormal physiological data and evaluating their utilities as feature extractors to enhance ML classifiers for predicting diagnoses, length of stay, and mortality. Furthermore, we examine prompt engineering techniques on zero-shot and few-shot LLM embeddings to measure their impact comprehensively. Although findings suggest the raw data features still prevails in medical ML tasks, zero-shot LLM embeddings demonstrate competitive results, suggesting a promising avenue for future research in medical applications., Comment: Under review

Asteroseismology of dwarf stars cooler than the Sun is very challenging due to the low amplitudes and rapid timescales of oscillations. Here, we present the asteroseismic detection of solar-like oscillations at 4-minute timescales ($\nu_{\mathrm{max}}\sim4300\mu$Hz) in the nearby K-dwarf $\sigma$ Draconis using extreme precision Doppler velocity observations from the Keck Planet Finder and 20-second cadence photometry from NASA's Transiting Exoplanet Survey Satellite. The star is the coolest dwarf star to date with both velocity and luminosity observations of solar-like oscillations, having amplitudes of $5.9\pm0.8\,$cm$\,\text{s}^{-1}$ and $0.8\pm0.2$ ppm, respectively. These measured values are in excellent agreement with established luminosity-velocity amplitude relations for oscillations and provide further evidence that mode amplitudes for stars with $T_{\mathrm{eff}}<\,5500\,$K diminish in scale following a $(L/M)^{1.5}$ relation. By modeling the star's oscillation frequencies from photometric data, we measure an asteroseismic age of $4.5\pm0.9\,\rm{(ran)} \pm 1.0\,\rm{(sys)}$ Gyr. The observations demonstrate the capability of next-generation spectrographs and precise space-based photometry to extend observational asteroseismology to nearby cool dwarfs, which are benchmarks for stellar astrophysics and prime targets for directly imaging planets using future space-based telescopes., Comment: In review in The Astrophysical Journal

Generative models have been showing potential for producing data in mass. This study explores the enhancement of clinical natural language processing performance by utilizing synthetic data generated from advanced language models. Promising results show feasible applications in such a high-stakes domain., Comment: submitted to review

Documentation burden is a major contributor to clinician burnout, which is rising nationally and is an urgent threat to our ability to care for patients. Artificial intelligence (AI) chatbots, such as ChatGPT, could reduce clinician burden by assisting with documentation. Although many hospitals are actively integrating such systems into electronic medical record systems, AI chatbots utility and impact on clinical decision-making have not been studied for this intended use. We are the first to examine the utility of large language models in assisting clinicians draft responses to patient questions. In our two-stage cross-sectional study, 6 oncologists responded to 100 realistic synthetic cancer patient scenarios and portal messages developed to reflect common medical situations, first manually, then with AI assistance. We find AI-assisted responses were longer, less readable, but provided acceptable drafts without edits 58% of time. AI assistance improved efficiency 77% of time, with low harm risk (82% safe). However, 7.7% unedited AI responses could severely harm. In 31% cases, physicians thought AI drafts were human-written. AI assistance led to more patient education recommendations, fewer clinical actions than manual responses. Results show promise for AI to improve clinician efficiency and patient care through assisting documentation, if used judiciously. Monitoring model outputs and human-AI interaction remains crucial for safe implementation., Comment: 4 figures and tables in main, submitted for review

Electronic Health Records (EHRs) and routine documentation practices play a vital role in patients' daily care, providing a holistic record of health, diagnoses, and treatment. However, complex and verbose EHR narratives overload healthcare providers, risking diagnostic inaccuracies. While Large Language Models (LLMs) have showcased their potential in diverse language tasks, their application in the healthcare arena needs to ensure the minimization of diagnostic errors and the prevention of patient harm. In this paper, we outline an innovative approach for augmenting the proficiency of LLMs in the realm of automated diagnosis generation, achieved through the incorporation of a medical knowledge graph (KG) and a novel graph model: Dr.Knows, inspired by the clinical diagnostic reasoning process. We derive the KG from the National Library of Medicine's Unified Medical Language System (UMLS), a robust repository of biomedical knowledge. Our method negates the need for pre-training and instead leverages the KG as an auxiliary instrument aiding in the interpretation and summarization of complex medical concepts. Using real-world hospital datasets, our experimental results demonstrate that the proposed approach of combining LLMs with KG has the potential to improve the accuracy of automated diagnosis generation. More importantly, our approach offers an explainable diagnostic pathway, edging us closer to the realization of AI-augmented diagnostic decision support systems., Comment: Under review

Large language models (LLMs) like ChatGPT have excited scientists across fields; in medicine, one source of excitement is the potential applications of LLMs trained on electronic health record (EHR) data. But there are tough questions we must first answer if health care institutions are interested in having LLMs trained on their own data; should they train an LLM from scratch or fine-tune it from an open-source model? For healthcare institutions with a predefined budget, what are the biggest LLMs they can afford? In this study, we take steps towards answering these questions with an analysis on dataset sizes, model sizes, and costs for LLM training using EHR data. This analysis provides a framework for thinking about these questions in terms of data scale, compute scale, and training budgets.

The Dark Energy Spectroscopic Instrument (DESI) is currently measuring the spectra of 40\,million galaxies and quasars, the largest such survey ever made to probe the nature of cosmological dark energy. The 4-meter Mayall telescope at Kitt Peak National Observatory has been adapted for DESI, including the construction of a 3.2-degree diameter prime focus corrector that focuses astronomical light onto a 0.8-meter diameter focal surface with excellent image quality over the DESI bandpass of 360-980nm. The wide-field corrector includes six lenses, as large as 1.1-meters in diameter and as heavy as 237\,kilograms, including two counter-rotating wedged lenses that correct for atmospheric dispersion over Zenith angles from 0 to 60 degrees. The lenses, cells, and barrel assembly all meet precise alignment tolerances on the order of tens of microns. The barrel alignment is maintained throughout a range of observing angles and temperature excursions in the Mayall dome by use of a hexapod, which is itself supported by a new cage, ring, and truss structure. In this paper we describe the design, fabrication, and performance of the new corrector and associated structure, focusing on how they meet DESI requirements. In particular we describe the prescription and specifications of the lenses, design choices and error budgeting of the barrel assembly, stray light mitigations, and integration and test at the Mayall telescope. We conclude with some validation highlights that demonstrate the successful corrector on-sky performance, and list some lessons learned during the multi-year fabrication phase., Comment: 68 pages, 56 figures, 22 tables. Submitted to the Astronomical Journal

The BioNLP Workshop 2023 initiated the launch of a shared task on Problem List Summarization (ProbSum) in January 2023. The aim of this shared task is to attract future research efforts in building NLP models for real-world diagnostic decision support applications, where a system generating relevant and accurate diagnoses will augment the healthcare providers decision-making process and improve the quality of care for patients. The goal for participants is to develop models that generated a list of diagnoses and problems using input from the daily care notes collected from the hospitalization of critically ill patients. Eight teams submitted their final systems to the shared task leaderboard. In this paper, we describe the tasks, datasets, evaluation metrics, and baseline systems. Additionally, the techniques and results of the evaluation of the different approaches tried by the participating teams are summarized., Comment: To appear in the Proceedings of the 5th BioNLP Workshop at ACL

Generative artificial intelligence (AI) is a promising direction for augmenting clinical diagnostic decision support and reducing diagnostic errors, a leading contributor to medical errors. To further the development of clinical AI systems, the Diagnostic Reasoning Benchmark (DR.BENCH) was introduced as a comprehensive generative AI framework, comprised of six tasks representing key components in clinical reasoning. We present a comparative analysis of in-domain versus out-of-domain language models as well as multi-task versus single task training with a focus on the problem summarization task in DR.BENCH (Gao et al., 2023). We demonstrate that a multi-task, clinically trained language model outperforms its general domain counterpart by a large margin, establishing a new state-of-the-art performance, with a ROUGE-L score of 28.55. This research underscores the value of domain-specific training for optimizing clinical diagnostic reasoning tasks., Comment: Accepted to the Proceedings of the 5th Clinical NLP Workshop at ACL

Radiotherapy (RT) toxicities can impair survival and quality-of-life, yet remain under-studied. Real-world evidence holds potential to improve our understanding of toxicities, but toxicity information is often only in clinical notes. We developed natural language processing (NLP) models to identify the presence and severity of esophagitis from notes of patients treated with thoracic RT. We fine-tuned statistical and pre-trained BERT-based models for three esophagitis classification tasks: Task 1) presence of esophagitis, Task 2) severe esophagitis or not, and Task 3) no esophagitis vs. grade 1 vs. grade 2-3. Transferability was tested on 345 notes from patients with esophageal cancer undergoing RT. Fine-tuning PubmedBERT yielded the best performance. The best macro-F1 was 0.92, 0.82, and 0.74 for Task 1, 2, and 3, respectively. Selecting the most informative note sections during fine-tuning improved macro-F1 by over 2% for all tasks. Silver-labeled data improved the macro-F1 by over 3% across all tasks. For the esophageal cancer notes, the best macro-F1 was 0.73, 0.74, and 0.65 for Task 1, 2, and 3, respectively, without additional fine-tuning. To our knowledge, this is the first effort to automatically extract esophagitis toxicity severity according to CTCAE guidelines from clinic notes. The promising performance provides proof-of-concept for NLP-based automated detailed toxicity monitoring in expanded domains., Comment: 17 pages, 6 tables, 1figure, submiting to JCO-CCI for review

Daily progress notes are common types in the electronic health record (EHR) where healthcare providers document the patient's daily progress and treatment plans. The EHR is designed to document all the care provided to patients, but it also enables note bloat with extraneous information that distracts from the diagnoses and treatment plans. Applications of natural language processing (NLP) in the EHR is a growing field with the majority of methods in information extraction. Few tasks use NLP methods for downstream diagnostic decision support. We introduced the 2022 National NLP Clinical Challenge (N2C2) Track 3: Progress Note Understanding - Assessment and Plan Reasoning as one step towards a new suite of tasks. The Assessment and Plan Reasoning task focuses on the most critical components of progress notes, Assessment and Plan subsections where health problems and diagnoses are contained. The goal of the task was to develop and evaluate NLP systems that automatically predict causal relations between the overall status of the patient contained in the Assessment section and its relation to each component of the Plan section which contains the diagnoses and treatment plans. The goal of the task was to identify and prioritize diagnoses as the first steps in diagnostic decision support to find the most relevant information in long documents like daily progress notes. We present the results of 2022 n2c2 Track 3 and provide a description of the data, evaluation, participation and system performance., Comment: To appear in Journal of Biomedical Informatics

We present laboratory characterization of kilo-pixel, filled backshort-under-grid (BUG) transition-edge sensor (TES) arrays developed for the Primordial Inflation Polarization ExploreR (PIPER) balloon-borne instrument. PIPER is designed to map the polarization of the CMB on the largest angular scales and characterize dust foregrounds by observing a large fraction of the sky in four frequency bands in the range 200 to 600 GHz. The BUG TES arrays are read out by planar SQUID-based time division multiplexer chips (2dMUX) of matching form factor and hybridized directly with the detector arrays through indium bump bonding. Here, we discuss the performance of the 2dMUX and present measurements of the TES transition temperature, thermal conductance, saturation power, and preliminary noise performance. The detectors achieve saturation power below 1 pW and phonon noise equivalent power (NEP) on the order of a few aW/rtHz. Detector performance is further verified through pre-flight tests in the integrated PIPER receiver, performed in an environment simulating balloon float conditions., Comment: 11 pages, 11 figures