Search

Adverse event (AE) extraction following COVID-19 vaccines from text data is crucial for monitoring and analyzing the safety profiles of immunizations. Traditional deep learning models are adept at learning intricate feature representations and dependencies in sequential data, but often require extensive labeled data. In contrast, large language models (LLMs) excel in understanding contextual information, but exhibit unstable performance on named entity recognition tasks, possibly due to their broad but unspecific training. This study aims to evaluate the effectiveness of LLMs and traditional deep learning models in AE extraction, and to assess the impact of ensembling these models on performance. In this study, we utilized reports and posts from the VAERS (n=621), Twitter (n=9,133), and Reddit (n=131) as our corpora. Our goal was to extract three types of entities: "vaccine", "shot", and "ae". We explored and fine-tuned (except GPT-4) multiple LLMs, including GPT-2, GPT-3.5, GPT-4, and Llama-2, as well as traditional deep learning models like RNN and BioBERT. To enhance performance, we created ensembles of the three models with the best performance. For evaluation, we used strict and relaxed F1 scores to evaluate the performance for each entity type, and micro-average F1 was used to assess the overall performance. The ensemble model achieved the highest performance in "vaccine", "shot", and "ae" with strict F1-scores of 0.878, 0.930, and 0.925, respectively, along with a micro-average score of 0.903. In conclusion, this study demonstrates the effectiveness and robustness of ensembling fine-tuned traditional deep learning models and LLMs, for extracting AE-related information. This study contributes to the advancement of biomedical natural language processing, providing valuable insights into improving AE extraction from text data for pharmacovigilance and public health surveillance.

In acupuncture therapy, the accurate location of acupoints is essential for its effectiveness. The advanced language understanding capabilities of large language models (LLMs) like Generative Pre-trained Transformers (GPT) present a significant opportunity for extracting relations related to acupoint locations from textual knowledge sources. This study aims to compare the performance of GPT with traditional deep learning models (Long Short-Term Memory (LSTM) and Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT)) in extracting acupoint-related location relations and assess the impact of pretraining and fine-tuning on GPT's performance. We utilized the World Health Organization Standard Acupuncture Point Locations in the Western Pacific Region (WHO Standard) as our corpus, which consists of descriptions of 361 acupoints. Five types of relations ('direction_of,' 'distance_of,' 'part_of,' 'near_acupoint,' and 'located_near') (n= 3,174) between acupoints were annotated. Five models were compared: BioBERT, LSTM, pre-trained GPT-3.5, fine-tuned GPT-3.5, as well as pre-trained GPT-4. Performance metrics included micro-average exact match precision, recall, and F1 scores. Our results demonstrate that fine-tuned GPT-3.5 consistently outperformed other models in F1 scores across all relation types. Overall, it achieved the highest micro-average F1 score of 0.92. This study underscores the effectiveness of LLMs like GPT in extracting relations related to acupoint locations, with implications for accurately modeling acupuncture knowledge and promoting standard implementation in acupuncture training and practice. The findings also contribute to advancing informatics applications in traditional and complementary medicine, showcasing the potential of LLMs in natural language processing.

Though Vaccines are instrumental in global health, mitigating infectious diseases and pandemic outbreaks, they can occasionally lead to adverse events (AEs). Recently, Large Language Models (LLMs) have shown promise in effectively identifying and cataloging AEs within clinical reports. Utilizing data from the Vaccine Adverse Event Reporting System (VAERS) from 1990 to 2016, this study particularly focuses on AEs to evaluate LLMs' capability for AE extraction. A variety of prevalent LLMs, including GPT-2, GPT-3 variants, GPT-4, and Llama 2, were evaluated using Influenza vaccine as a use case. The fine-tuned GPT 3.5 model (AE-GPT) stood out with a 0.704 averaged micro F1 score for strict match and 0.816 for relaxed match. The encouraging performance of the AE-GPT underscores LLMs' potential in processing medical data, indicating a significant stride towards advanced AE detection, thus presumably generalizable to other AE extraction tasks.

Background: Identifying relationships between clinical events and temporal expressions is a key challenge in meaningfully analyzing clinical text for use in advanced AI applications. While previous studies exist, the state-of-the-art performance has significant room for improvement. Methods: We studied several variants of BERT (Bidirectional Encoder Representations using Transformers) some involving clinical domain customization and the others involving improved architecture and/or training strategies. We evaluated these methods using a direct temporal relations dataset which is a semantically focused subset of the 2012 i2b2 temporal relations challenge dataset. Results: Our results show that RoBERTa, which employs better pre-training strategies including using 10x larger corpus, has improved overall F measure by 0.0864 absolute score (on the 1.00 scale) and thus reducing the error rate by 24% relative to the previous state-of-the-art performance achieved with an SVM (support vector machine) model. Conclusion: Modern contextual language modeling neural networks, pre-trained on a large corpus, achieve impressive performance even on highly-nuanced clinical temporal relation tasks., Comment: 10 pages, 1 Figure, 7 Tables

In this work, we demonstrate a remarkable change of chemical trend of Iron under high pressure that is of great importance for understanding the distribution of elements in the Earth's mantle and core. Using first principles crystal structure search method, we conduct a systematic study of the propensity of p block elements to chemically bind with iron under high pressures ranging from ambient conditions to that of Earth's core. We show that under increasing pressure, iron tends to reverse its chemical nature, changing from an electron donor (reductant) to an electron acceptor, and oxidizes p-block elements in many compounds. Such reverse chemistry has a significant impact on the stoichiometries, bond types and strengths, structures and properties of iron compounds under deep planetary conditions., Comment: 21 pages, 11 figures

To obtain high-performance spintronic devices with high integration density, two-dimensional (2D) half-metallic materials are eagerly pursued all along. Here, we propose a stable 2D material with a honeycomb-kagome lattice, i.e., the Mg3C2 monolayer, based on first-principles calculations. This monolayer is an anti-ferromagnetic (AFM) semiconductor at its ground state. We further demonstrate that a transition from AFM semiconductor to ferromagnetic half-metal in this 2D material can be induced by carrier (electron or hole) doping. This magnetic transition can be understood by the Stoner criterion. In addition, the half-metallicity arises from the 2pz orbitals of the carbon (C) atoms for the electron-doped system, but from the C 2px and 2py orbitals for the case of hole doping. Our findings highlight a new promising material with controllable magnetic and electronic properties toward 2D spintronic applications., Comment: 6 figures,20 pages

Searching for two-dimensional (2D) organic Dirac materials, which have more adaptable practical applications in comparing with inorganic ones, is of great significance and has been ongoing. However, only two kinds of these materials with low Fermi velocity have been discovered so far. Herein, we report the design of an organic monolayer with C$_4$N$_3$H stoichiometry which possesses fascinating structure and good stability in its free-standing state. More importantly, we demonstrate that this monolayer is a semimetal with anisotropic Dirac cones and very high Fermi velocity. This Fermi velocity is roughly one order of magnitude larger than that in 2D organic Dirac materials ever reported, and is comparable to that in graphene. The Dirac states in this monolayer arise from the extended $\pi$-electron conjugation system formed by the overlapping 2\emph{p}$_z$ orbitals of carbon and nitrogen atoms. Our finding opens a door for searching more 2D organic Dirac materials with high Fermi velocity., Comment: 8 pages, 6 figures