Search

Among the various molecular properties and their combinations, it is a costly process to obtain the desired molecular properties through theory or experiment. Using machine learning to analyze molecular structure features and to predict molecular properties is a potentially efficient alternative for accelerating the prediction of molecular properties. In this study, we analyze molecular properties through the molecular structure from the perspective of machine learning. We use SMILES sequences as inputs to an artificial neural network in extracting molecular structural features and predicting molecular properties. A SMILES sequence comprises symbols representing molecular structures. To address the problem that a SMILES sequence is different from actual molecular structural data, we propose a pretraining model for a SMILES sequence based on the BERT model, which is widely used in natural language processing, such that the model learns to extract the molecular structural information contained in the SMILES sequence. In an experiment, we first pretrain the proposed model with 100,000 SMILES sequences and then use the pretrained model to predict molecular properties on 22 data sets and the odor characteristics of molecules (98 types of odor descriptor). The experimental results show that our proposed pretraining model effectively improves the performance of molecular property prediction Scientific contribution: The 2-encoder pretraining is proposed by focusing on the lower dependency of symbols to the contextual environment in a SMILES than one in a natural language sentence and the corresponding of one compound to multiple SMILES sequences. The model pretrained with 2-encoder shows higher robustness in tasks of molecular properties prediction compared to BERT which is adept at natural language. [ABSTRACT FROM AUTHOR]

Male infertility has affected an increasingly large population over the past few decades, affecting over 186 million people globally. The advent of assisted reproductive technologies (ARTs) and artificial intelligence (AI) has changed the landscape of diagnosis and treatment of male infertility. Through an extensive literature review encompassing the PubMed, Google Scholar, and Scopus databases, various AI techniques such as machine learning (ML), artificial neural networks (ANNs), deep learning (DL), and natural language processing (NLP) were examined in the context of evaluating seminal quality, predicting fertility potential, and improving semen analysis. Research indicates that AI models can accurately estimate the quality of semen, diagnose problems with sperm, and provide guidance on reproductive health decisions. In addition, developments in smartphone-based semen analyzers and computer-assisted semen analysis (CASA) are indicative of initiatives to improve the price, portability, and accuracy of results. Future directions point to possible uses for AI in ultrasonography assessment, microsurgical testicular sperm extraction (microTESE), and home-based semen analysis. Overall, AI holds significant promise in revolutionizing the diagnosis and treatment of male infertility, offering standardized, objective, and efficient approaches to addressing this global health challenge. [ABSTRACT FROM AUTHOR]

Language in nonmedical data sets is known to transmit human-like biases when used in natural language processing (NLP) algorithms that can reinforce disparities. It is unclear if NLP algorithms of medical notes could lead to similar transmissions of biases. Can we identify implicit bias in clinical notes, and are biases stable across time and geography? To determine whether different racial and ethnic descriptors are similar contextually to stigmatizing language in ICU notes and whether these relationships are stable across time and geography, we identified notes on critically ill adults admitted to the University of California, San Francisco (UCSF), from 2012 through 2022 and to Beth Israel Deaconess Hospital (BIDMC) from 2001 through 2012. Because word meaning is derived largely from context, we trained unsupervised word-embedding algorithms to measure the similarity (cosine similarity) quantitatively of the context between a racial or ethnic descriptor (eg, African-American) and a stigmatizing target word (eg, nonco - operative) or group of words (violence , passivity , noncompliance , nonadherence). In UCSF notes, Black descriptors were less likely to be similar contextually to violent words compared with White descriptors. Contrastingly, in BIDMC notes, Black descriptors were more likely to be similar contextually to violent words compared with White descriptors. The UCSF data set also showed that Black descriptors were more similar contextually to passivity and noncompliance words compared with Latinx descriptors. Implicit bias is identifiable in ICU notes. Racial and ethnic group descriptors carry different contextual relationships to stigmatizing words, depending on when and where notes were written. Because NLP models seem able to transmit implicit bias from training data, use of NLP algorithms in clinical prediction could reinforce disparities. Active debiasing strategies may be necessary to achieve algorithmic fairness when using language models in clinical research. [Display omitted] [ABSTRACT FROM AUTHOR]

Named entity recognition (NER) plays a pivotal role in Natural Language Processing by identifying and classifying entities within textual data. While NER methodologies have seen significant advancements, driven by pretrained word embeddings and deep neural networks, the majority of these studies have focused on text with well-defined grammar and structure. A significant research gap exists concerning NER in informal or unstructured text, where traditional grammar rules and sentence structure are absent. This research addresses this crucial gap by focusing on the detection of product names within unstructured Turkish text. To accomplish this, we propose a deep learning-based NER model which combines a Bidirectional Long Short-Term Memory (BiLSTM) architecture with a Conditional Random Field (CRF) layer, further enhanced by FastText embeddings. To comprehensively evaluate and compare our model's performance, we explore different embedding approaches, including Word2Vec and Glove, in conjunction with the Bidirectional Long Short-Term Memory and Conditional Random Field (BiLSTM-CRF) model. Furthermore, we conduct comparisons against BERT to assess the efficacy of our approach. Our experimentation utilizes a Turkish e-commerce dataset gathered from the internet, where traditional grammatical and structural rules may not apply. The BiLSTM-CRF model with FastText embeddings achieved an F1 score value of 57.40%, a precision value of 55.78%, and a recall value of 59.12%. These results indicate promising performance in outperforming other baseline techniques. This research contributes to the field of NER by addressing the unique challenges posed by unstructured Turkish text and opens avenues for improved entity recognition in informal language settings, with potential applications across various domains. [ABSTRACT FROM AUTHOR]

Several hardware Trojan (HT) detection techniques are available today to ensure the security of hardware systems. However, the existing pre-silicon HT detection techniques have problems such as difficulties in capturing HT path features and poor applicability. To address these challenges, this paper proposes a gate-level HT detection scheme based on a deep learning model. We parse the circuit gate-level netlist and develop an algorithm to extract circuit path sentences based on the signal propagation rule. Path sentences consisting of gate names are extracted as experimental datasets. We apply the theory of natural language processing (NLP) to the task of HT detection and use three neural networks to filter the length of path sentences. Then, based on the deep learning model text convolutional neural network (TextCNN), we propose PS-TextCNN for HT detection. Our approach is verified on seven benchmark circuits of the RS232-series and eight benchmark circuits of the s-series. We achieve an average true positive rate (TPR) of 88.9%. The TPR of the RS232-series reaches a high score of 99.5%. The TPR of the s-series is 79.5%, which is significantly higher than that of the existing gate-level HT detection techniques. [ABSTRACT FROM AUTHOR]

Deep neural network has made surprising achievements in natural language processing, image pattern classification recognition, and other domains in the last few years. It is still tough to apply to hardware-constrained or mobile equipment because of the huge number of parameters, high storage as well as computing costs. In this paper, a new sparse iteration neural network architecture is proposed. First, the pruning method is used to compress the model size and make the network sparse. Then the architecture is iterated on the sparse network model, and the network performance is improved without adding additional parameters. Finally, the hybrid deep learning model was carried out on CV tasks and NLP tasks on ANN, CNN, and Transformer. Compared with the sparse network architecture, we finally found that the accuracy of the MINST, CIFAR10, PASCAL VOC 2012, and SQuAD datasets is improved by 0.47%, 0.64%, 3.75%, and 15.06%, respectively. [ABSTRACT FROM AUTHOR]

This document is a letter to the editor discussing the potential of generative artificial intelligence (AI) and large language models (LLMs) to contribute to value-based healthcare. The letter highlights recent regulatory actions and advancements in AI technology, such as deep learning and transformer LLMs. It emphasizes the need for clinical leadership and alignment between AI policy and healthcare goals. The author suggests that AI systems should support human intelligence and decision-making while prioritizing patient-centered care. [Extracted from the article]

Predicting interactions between microbes and hosts plays critical roles in microbiome population genetics and microbial ecology and evolution. How to systematically characterize the sophisticated mechanisms and signal interplay between microbes and hosts is a significant challenge for global health risks. Identifying microbe-host interactions (MHIs) can not only provide helpful insights into their fundamental regulatory mechanisms, but also facilitate the development of targeted therapies for microbial infections. In recent years, computational methods have become an appealing alternative due to the high risk and cost of wet-lab experiments. Therefore, in this study, we utilized rich microbial metagenomic information to construct a novel heterogeneous microbial network (HMN)-based model named KGVHI to predict candidate microbes for target hosts. Specifically, KGVHI first built a HMN by integrating human proteins, viruses and pathogenic bacteria with their biological attributes. Then KGVHI adopted a knowledge graph embedding strategy to capture the global topological structure information of the whole network. A natural language processing algorithm is used to extract the local biological attribute information from the nodes in HMN. Finally, we combined the local and global information and fed it into a blended deep neural network (DNN) for training and prediction. Compared to state-of-the-art methods, the comprehensive experimental results show that our model can obtain excellent results on the corresponding three MHI datasets. Furthermore, we also conducted two pathogenic bacteria case studies to further indicate that KGVHI has excellent predictive capabilities for potential MHI pairs. [ABSTRACT FROM AUTHOR]

The implementation of Artificial Intelligence (AI) in healthcare is enhancing diagnostic accuracy in clinical setups. The use of AI in healthcare is steadily increasing with advancing technology, extending beyond disease diagnosis to encompass roles in feto-maternal health. AI harnesses Machine Learning (ML), Natural Language Processing (NLP), Artificial Neural Networks (ANN), and computer vision to analyze data and draw conclusions. Considering maternal health, ML analyzes vast datasets to predict maternal and fetal health outcomes, while NLP interprets medical texts and patient records to assist in diagnosis and treatment decisions. ANN models identify patterns in complex feto-maternal medical data, aiding in risk assessment and intervention planning whereas, computer vision enables the analysis of medical images for early detection of feto-maternal complications. AI facilitates early pregnancy detection, genetic screening, and continuous monitoring of maternal health parameters, providing real-time alerts for deviations, while also playing a crucial role in the early detection of fetal abnormalities through enhanced ultrasound imaging, contributing to informed decision-making. This review investigates into the application of AI, particularly through predictive models, in addressing the monitoring of feto-maternal health. Additionally, it examines potential future directions and challenges associated with these applications. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has revolutionized natural language processing (NLP) by enabling the development of models that can learn complex representations of language data, leading to significant improvements in performance across a wide range of NLP tasks. Deep learning models for NLP typically use large amounts of data to train deep neural networks, allowing them to learn the patterns and relationships in language data. This is in contrast to traditional NLP approaches, which rely on hand-engineered features and rules to perform NLP tasks. The ability of deep neural networks to learn hierarchical representations of language data, handle variable-length input sequences, and perform well on large datasets makes them well-suited for NLP applications. Driven by the exponential growth of textual data and the increasing demand for condensed, coherent, and informative summaries, text summarization has been a critical research area in the field of NLP. Applying deep learning to text summarization refers to the use of deep neural networks to perform text summarization tasks. In this survey, we begin with a review of fashionable text summarization tasks in recent years, including extractive, abstractive, multi-document, and so on. Next, we discuss most deep learning-based models and their experimental results on these tasks. The paper also covers datasets and data representation for summarization tasks. Finally, we delve into the opportunities and challenges associated with summarization tasks and their corresponding methodologies, aiming to inspire future research efforts to advance the field further. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific setting. This survey aims to provide a comprehensive review of existing techniques, evaluation methodologies, and practical applications of automatic text summarization. [ABSTRACT FROM AUTHOR]

Given the foundational role of system requirements in design projects, designers can benefit from classifying, comparing, and observing connections between requirements. Manually undertaking these processes, however, can be laborious and time-consuming. Previous studies have employed Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art natural language processing (NLP) deep neural network model, to automatically analyze written requirements. Yet, it remains unclear whether BERT can sufficiently capture the nuances that differentiate requirements between and within design documents. This work evaluates BERT's performance on two requirement classification tasks (one inter-document and one intra-document) executed on a corpus of 1,303 requirements sourced from five system design projects. First, in the "parent document classification" task, a BERT model is fine-tuned to classify requirements according to their originating project. A separate BERT model is then fine-tuned on a "functional classification" task where each requirement is classified as either functional or nonfunctional. Our results also include a comparison with a baseline model, Word2Vec, and demonstrate that our model achieves higher classification accuracy. When evaluated on test sets, the former model receives a Matthews correlation coefficient (MCC) of 0.95, while the latter receives an MCC of 0.82, indicating BERT's ability to reliably distinguish requirements. This work then explores the application of BERT's representations, known as embeddings, to identify similar requirements and predict requirement change. [ABSTRACT FROM AUTHOR]

The rapid progress of AI impacts diverse scientific disciplines, including toxicology, and has the potential to transform chemical safety evaluation. Toxicology has evolved from an empirical science focused on observing apical outcomes of chemical exposure, to a data-rich field ripe for AI integration. The volume, variety and velocity of toxicological data from legacy studies, literature, high-throughput assays, sensor technologies and omics approaches create opportunities but also complexities that AI can help address. In particular, machine learning is well suited to handle and integrate large, heterogeneous datasets that are both structured and unstructured—a key challenge in modern toxicology. AI methods like deep neural networks, large language models, and natural language processing have successfully predicted toxicity endpoints, analyzed high-throughput data, extracted facts from literature, and generated synthetic data. Beyond automating data capture, analysis, and prediction, AI techniques show promise for accelerating quantitative risk assessment by providing probabilistic outputs to capture uncertainties. AI also enables explanation methods to unravel mechanisms and increase trust in modeled predictions. However, issues like model interpretability, data biases, and transparency currently limit regulatory endorsement of AI. Multidisciplinary collaboration is needed to ensure development of interpretable, robust, and human-centered AI systems. Rather than just automating human tasks at scale, transformative AI can catalyze innovation in how evidence is gathered, data are generated, hypotheses are formed and tested, and tasks are performed to usher new paradigms in chemical safety assessment. Used judiciously, AI has immense potential to advance toxicology into a more predictive, mechanism-based, and evidence-integrated scientific discipline to better safeguard human and environmental wellbeing across diverse populations. [ABSTRACT FROM AUTHOR]

Machine learning techniques are extensively employed in drug discovery, with a significant focus on developing QSAR models that interpret the structural information of potential drugs. In this study, the pre-trained natural language processing (NLP) model, ChemBERTa, was utilized in the drug discovery process. We proposed and evaluated four core model architectures as follows: deep neural network (DNN), encoder, concatenation (concat), and pipe. The DNN model processes physicochemical properties as input, while the encoder model leverages the simplified molecular input line entry system (SMILES) along with NLP techniques. The latter two models, concat and pipe, incorporate both SMILES and physicochemical properties, operating in parallel and with sequential manners, respectively. We collected 5238 entries from DrugBank, including their physicochemical properties and absorption, distribution, metabolism, excretion, and toxicity (ADMET) features. The models' performance was assessed by the area under the receiver operating characteristic curve (AUROC), with the DNN, encoder, concat, and pipe models achieved 62.4%, 76.0%, 74.9%, and 68.2%, respectively. In a separate test with 84 experimental microsomal stability datasets, the AUROC scores for external data were 78% for DNN, 44% for the encoder, and 50% for concat, indicating that the DNN model had superior predictive capabilities for new data. This suggests that models based on structural information may require further optimization or alternative tokenization strategies. The application of natural language processing techniques to pharmaceutical challenges has demonstrated promising results, highlighting the need for more extensive data to enhance model generalization. [ABSTRACT FROM AUTHOR]

The evolution of a turbulent flow to a statistically steady state can be cast as a multiscale problem involving energy redistribution processes that take place on the long, large eddy turnover timescale and chaotic processes that take place on the much shorter timescale of the turbulence fluctuations. But the absence of a way to perform super-resolution reconstructions of the instantaneous velocity field from its lower-dimensional moments has prevented the use of standard multiscale computational approaches for accelerating turbulence simulations. We introduce an encoder-decoder recurrent neural network model, an architecture typically used in natural language processing, for translating between the instantaneous velocity field and energy spectrum in a one-dimensional turbulent flow. We use the model in a multiscale simulation scheme to evolve the stochastic Burgers' equation and recover the final, statistically stationary turbulent Burgers' velocity field up to 443 times faster in wall-clock time than using direct numerical simulation alone. [ABSTRACT FROM AUTHOR]

The given text discusses a road segmentation method called MSAN that uses multi-modal scale-aware attentions to segment roads in real-time for autonomous driving. The method combines RGB and depth information to improve prediction results and outperforms single modality models. The MSAN architecture consists of two attention modules, one for feature fusion across modalities and scales and another for scale-aware fusion. The experiments show that MSAN achieves high performance and efficiency on multiple datasets. However, the method relies heavily on labeled data, which limits its adaptability to different scenarios. Future research aims to reduce data dependency through few-shot learning strategies. [Extracted from the article]

The recent evolution of artificial intelligence (AI) can be considered life‐changing. In particular, there is great interest in emerging hot topics in AI such as image classification and natural language processing. Our world has been revolutionized by convolutional neural networks and transformer for image classification and natural language processing, respectively. Moreover, these techniques can be used in the field of dementia. We introduce some applications of AI systems for treating and diagnosing dementia, including image‐classification AI for recognizing facial features associated with dementia, image‐classification AI for classifying leukoaraiosis in MRI images, object‐detection AI for detecting microbleeding in MRI images, object‐detection AI for support care, natural language‐processing AI for detecting dementia within conversations, and natural language‐processing AI for chatbots. Such AI technologies can significantly transform the future of dementia diagnosis and treatment. Geriatr Gerontol Int 2024; 24: 25–30. [ABSTRACT FROM AUTHOR]

Deep neural networks are extensively used in text mining and Natural Language Processing is to enable computers to understand, analyze, and generate natural language data, such as text or speech, but semantic resources, such as taxonomies and ontologies, are not fully included in deep learning. In this paper, we use Deep Convolutional Neural Network (Deep CNN) to classify research papers using the Computer Science Ontology, an ontology of research areas in the field of computer science. It takes as input the abstract and keywords of a particular research paper and returns the relevant research topic. To evaluate our ontology, we used a gold standard dataset that includes research articles. To further improve text classification results, we propose to design a Deep CNN model. We then used ontology matching to reduce the classes and get better results. Experimental results show that the proposed approach outperforms the one with the highest precision, recall, and F1-score. [ABSTRACT FROM AUTHOR]

Connections between circular RNAs (circRNAs) and microRNAs (miRNAs) assume a pivotal position in the onset, evolution, diagnosis and treatment of diseases and tumors. Selecting the most potential circRNA-related miRNAs and taking advantage of them as the biological markers or drug targets could be conducive to dealing with complex human diseases through preventive strategies, diagnostic procedures and therapeutic approaches. Compared to traditional biological experiments, leveraging computational models to integrate diverse biological data in order to infer potential associations proves to be a more efficient and cost-effective approach. This paper developed a model of Convolutional Autoencoder for CircRNA–MiRNA Associations (CA-CMA) prediction. Initially, this model merged the natural language characteristics of the circRNA and miRNA sequence with the features of circRNA–miRNA interactions. Subsequently, it utilized all circRNA–miRNA pairs to construct a molecular association network, which was then fine-tuned by labeled samples to optimize the network parameters. Finally, the prediction outcome is obtained by utilizing the deep neural networks classifier. This model innovatively combines the likelihood objective that preserves the neighborhood through optimization, to learn the continuous feature representation of words and preserve the spatial information of two-dimensional signals. During the process of 5-fold cross-validation, CA-CMA exhibited exceptional performance compared to numerous prior computational approaches, as evidenced by its mean area under the receiver operating characteristic curve of 0.9138 and a minimal SD of 0.0024. Furthermore, recent literature has confirmed the accuracy of 25 out of the top 30 circRNA–miRNA pairs identified with the highest CA-CMA scores during case studies. The results of these experiments highlight the robustness and versatility of our model. [ABSTRACT FROM AUTHOR]

Traditional Chinese medicine (TCM) has been used for thousands of years and has been proven to be effective at treating many complicated illnesses with minimal side effects. The application and advancement of TCM are, however, constrained by the absence of objective measuring standards due to its relatively abstract diagnostic methods and syndrome differentiation theories. Ongoing developments in machine learning (ML) and deep learning (DL), specifically in computer vision (CV) and natural language processing (NLP), offer novel opportunities to modernize TCM by exploring the profound connotations of its theory. This review begins with an overview of the ML and DL methods employed in TCM; this is followed by practical instances of these applications. Furthermore, extensive discussions emphasize the mature integration of ML and DL in TCM, such as tongue diagnosis, pulse diagnosis, and syndrome differentiation treatment, highlighting their early successful application in the TCM field. Finally, this study validates the accomplishments and addresses the problems and challenges posed by the application and development of TCM powered by ML and DL. As ML and DL techniques continue to evolve, modern technology will spark new advances in TCM. [ABSTRACT FROM AUTHOR]

Simple Summary: Transformer models, originally successful in natural language processing, have found application in computer vision, demonstrating promising results in tasks related to cancer image analysis. Despite being one of the prevalent and swiftly spreading cancers globally, there is a pressing need for accurate automated analysis methods for oral cancer. This need is particularly critical for high-risk populations residing in low- and middle-income countries. In this study, we evaluated the performance of the Vision Transformer (ViT) and the Swin Transformer in the classification of mobile-based oral cancer images we collected from high-risk populations. The results showed that the Swin Transformer model achieved higher accuracy than the ViT model, and both transformer models work better than the conventional convolution model VGG19. Oral cancer, a pervasive and rapidly growing malignant disease, poses a significant global health concern. Early and accurate diagnosis is pivotal for improving patient outcomes. Automatic diagnosis methods based on artificial intelligence have shown promising results in the oral cancer field, but the accuracy still needs to be improved for realistic diagnostic scenarios. Vision Transformers (ViT) have outperformed learning CNN models recently in many computer vision benchmark tasks. This study explores the effectiveness of the Vision Transformer and the Swin Transformer, two cutting-edge variants of the transformer architecture, for the mobile-based oral cancer image classification application. The pre-trained Swin transformer model achieved 88.7% accuracy in the binary classification task, outperforming the ViT model by 2.3%, while the conventional convolutional network model VGG19 and ResNet50 achieved 85.2% and 84.5% accuracy. Our experiments demonstrate that these transformer-based architectures outperform traditional convolutional neural networks in terms of oral cancer image classification, and underscore the potential of the ViT and the Swin Transformer in advancing the state of the art in oral cancer image analysis. [ABSTRACT FROM AUTHOR]

Much recent effort has been devoted to creating large-scale language models. Nowadays, the most prominent approaches are based on deep neural networks, such as BERT. However, they lack transparency and interpretability, and are often seen as black boxes. This affects not only their applicability in downstream tasks but also the comparability of different architectures or even of the same model trained using different corpora or hyperparameters. In this paper, we propose a set of intrinsic evaluation tasks that inspect the linguistic information encoded in models developed for Brazilian Portuguese. These tasks are designed to evaluate how different language models generalise information related to grammatical structures and multiword expressions (MWEs), thus allowing for an assessment of whether the model has learned different linguistic phenomena. The dataset that was developed for these tasks is composed of a series of sentences with a single masked word and a cue phrase that helps in narrowing down the context. This dataset is divided into MWEs and grammatical structures, and the latter is subdivided into 6 tasks: impersonal verbs, subject agreement, verb agreement, nominal agreement, passive and connectors. The subset for MWEs was used to test BERTimbau Large, BERTimbau Base and mBERT. For the grammatical structures, we used only BERTimbau Large, because it yielded the best results in the MWE task. In both cases, we evaluated the results considering the best candidates and the top ten candidates. The evaluation was done both automatically (for MWEs) and manually (for grammatical structures). The results obtained for MWEs show that BERTimbau Large surpassed both the other models in predicting the correct masked element. However, the average accuracy of the best model was only 52% when only the best candidates were considered for each sentence, going up to 66% when the top ten candidates were taken into account. As for the grammatical tasks, the results presented better prediction, but also varied depending on the type of morphosyntactic agreement. On the one hand, cases such as connectors and impersonal verbs, which do not require any agreement in the produced candidates, had precision of 100% and 98.78% among the best candidates. On the other hand, tasks that require morphosyntactic agreement had results consistently below 90% overall precision, with the lowest scores being reported for nominal agreement and verb agreement, both having scores below 80% in overall precision among the best candidates. Therefore, we identified that a critical and widely adopted resource for Brazilian Portuguese NLP presents issues concerning MWE vocabulary and morphosyntactic agreement, even if it is prolific in most cases. These models are a core component in many NLP systems, and our findings demonstrate the need of additional improvements in these models and the importance of widely evaluating computational representations of language. [ABSTRACT FROM AUTHOR]

This paper introduces an effective method for improving dependency parsing which is based on a graph embedding model. The model helps extract local and global connectivity patterns between tokens. This method allows neural network models to perform better on dependency parsing benchmarks. We propose to incorporate node embeddings trained by a graph embedding algorithm into a bidirectional recurrent neural network scheme. The new model outperforms a baseline reference using a state-of-the-art method on three dependency treebanks for both low-resource and high-resource natural languages, namely Indonesian, Vietnamese and English. We also show that the popular pretraining technique of BERT would not pick up on the same kind of signal as graph embeddings. The new parser together with all trained models is made available under an open-source license, facilitating community engagement and advancement of natural language processing research for two low-resource languages with around 300 million users worldwide in total. [ABSTRACT FROM AUTHOR]

Within the domain of Natural Language Processing (NLP), Large Language Models (LLMs) represent sophisticated models engineered to comprehend, generate, and manipulate text resembling human language on an extensive scale. They are transformer-based deep learning architectures, obtained through the scaling of model size, pretraining of corpora, and computational resources. The potential healthcare applications of these models primarily involve chatbots and interaction systems for clinical documentation management, and medical literature summarization (Biomedical NLP). The challenge in this field lies in the research for applications in diagnostic and clinical decision support, as well as patient triage. Therefore, LLMs can be used for multiple tasks within patient care, research, and education. Throughout 2023, there has been an escalation in the release of LLMs, some of which are applicable in the healthcare domain. This remarkable output is largely the effect of the customization of pre-trained models for applications like chatbots, virtual assistants, or any system requiring human-like conversational engagement. As healthcare professionals, we recognize the imperative to stay at the forefront of knowledge. However, keeping abreast of the rapid evolution of this technology is practically unattainable, and, above all, understanding its potential applications and limitations remains a subject of ongoing debate. Consequently, this article aims to provide a succinct overview of the recently released LLMs, emphasizing their potential use in the field of medicine. Perspectives for a more extensive range of safe and effective applications are also discussed. The upcoming evolutionary leap involves the transition from an AI-powered model primarily designed for answering medical questions to a more versatile and practical tool for healthcare providers such as generalist biomedical AI systems for multimodal-based calibrated decision-making processes. On the other hand, the development of more accurate virtual clinical partners could enhance patient engagement, offering personalized support, and improving chronic disease management. [ABSTRACT FROM AUTHOR]

Currently, direct surveys are used less and less to assess satisfaction with the quality of user services. One of the most effective methods to solve this problem is to extract user attitudes from social media texts using natural language text mining. This approach helps to obtain more objective results by increasing the representativeness and independence of the sample of service consumers being studied. The purpose of this article is to improve existing methods and test a method for classifying Russian-language text reviews of patients about the work of medical institutions and doctors, extracted from social media resources. The authors developed a hybrid method for classifying text reviews about the work of medical institutions and tested machine learning methods using various neural network architectures (GRU, LSTM, CNN) to achieve this goal. More than 60,000 reviews posted by patients on the two most popular doctor review sites in Russia were analysed. Main results: (1) the developed classification algorithm is highly efficient—the best result was shown by the GRU-based architecture (val_accuracy = 0.9271); (2) the application of the method of searching for named entities to text messages after their division made it possible to increase the classification efficiency for each of the classifiers based on the use of artificial neural networks. This study has scientific novelty and practical significance in the field of social and demographic research. To improve the quality of classification, in the future, it is planned to expand the semantic division of the review by object of appeal and sentiment and take into account the resulting fragments separately from each other. [ABSTRACT FROM AUTHOR]

Data mining and natural language processing researchers have been working on sentiment analysis for the past decade. Using deep neural networks (DNNs) for sentiment analysis has recently shown promising results. A technique of studying people's attitudes through emotional sentiment analysis of data generated from various sources such as Twitter, social media reviews, etc. and classifying emotions based on the given data is related to text data generation. Therefore, the proposed study proposes a well-known deep learning technique for facet-based emotional mood classification using text data that can handle a large amount of content. Text data pre-processing uses stemming, segmentation, tokenization, case folding, and removal of stop words, nulls, and special characters. After data pre-processing, three word embedding approaches such as Assimilated N-gram Approach (ANA), Boosted Term Frequency Inverse Document Frequency (BT-IDF) and Enhanced Two-Way Encoder Representation from Transformers (E-BERT) are used to extract relevant features. The extracted features from the three different approaches are concatenated using the Feature Fusion Approach (FFA). The optimal features are selected using the Intensified Hunger Games Search Optimization (I-HGSO) algorithm. Finally, aspect-based sentiment analysis is performed using the Senti-BILSTM (Deep Aspect-EMO SentiNet) autoencoder based on the Hybrid Emotional Aspect Capsule autoencoder. The experiment was built on the yelp reviews dataset, IDMB movie review dataset, Amazon reviews dataset and the Twitter sentiment dataset. A statistical evaluation and comparison of the experimental results are conducted with respect to the accuracy, precision, specificity, the f1-score, recall, and sensitivity. There is a 99.26% accuracy value in the Yelp reviews dataset, a 99.46% accuracy value in the IMDB movie reviews dataset, a 99.26% accuracy value in the Amazon reviews dataset and a 99.93% accuracy value in the Twitter sentiment dataset. [ABSTRACT FROM AUTHOR]

Uncertainty quantification (UQ) in machine learning is currently drawing increasing research interest, driven by the rapid deployment of deep neural networks across different fields, such as computer vision and natural language processing, and by the need for reliable tools in risk-sensitive applications. Recently, various machine learning models have also been developed to tackle problems in the field of scientific computing with applications to computational science and engineering (CSE). Physics-informed neural networks and deep operator networks are two such models for solving partial differential equations (PDEs) and learning operator mappings, respectively. In this regard, a comprehensive study of UQ methods tailored specifically for scientific machine learning (SciML) models has been provided in [A. F. Psaros et al., J. Comput. Phys., 477 (2023), art. 111902]. Nevertheless, and despite their theoretical merit, implementations of these methods are not straightforward, especially in large-scale CSE applications, hindering their broad adoption in both research and industry settings. In this paper, we present an open-source Python library (https://github.com/Crunch-UQ4MI), termed NeuralUQ and accompanied by an educational tutorial, for employing UQ methods for SciML in a convenient and structured manner. The library, designed for both educational and research purposes, supports multiple modern UQ methods and SciML models. It is based on a succinct workflow and facilitates flexible employment and easy extensions by the users. We first present a tutorial of NeuralUQ and subsequently demonstrate its applicability and efficiency in four diverse examples, involving dynamical systems and high-dimensional parametric and time-dependent PDEs. [ABSTRACT FROM AUTHOR]

Semantic relatedness is one of the most significant aspects of natural language processing. It has been identified as a critical technology for developing intelligent systems like Siri, Microsoft Ice, Cortana, and Xiaoai. In 2014, SemEval ranked SR as the top task. While many existing studies have focused on analyzing the entailment of single phrases, advancements in deep learning have made it possible to analyze complete sentences or texts. While the natural parallelism of membrane computing has shown promise for data processing, harnessing this potential to advance semantic relatedness remains an open problem yet to be tackled. This paper proposes a novel Siamese Manhattan LSTM-SNP approach (SiMaLSTM-SNP) for the SR problem. The approach uses a collaborative Word2vec and 10-Layer Attention strategy to represent and extract sentence pairs and a Siamese LSTM-SNP structure to calculate the hidden states of sentences. The multi-head self-attention layer identifies text associations and redistributes hidden state weights. The last hidden state is extracted, and the relatedness score is calculated using the Manhattan distance. The experiments demonstrate that SiMaLSTM-SNP outperforms 17 classical SR baselines and 7 novel approaches on the standard datasets SICK and STS in terms of mean square error performance. This indicates that SiMaLSTM-SNP can accurately capture the semantic distinction between two sentences and effectively preserve their semantic information. [ABSTRACT FROM AUTHOR]

Negation is a linguistic phenomenon that can cause sentences to have their meanings reversed. It frequently inverts affirmative sentences into negative ones, affecting the polarity; therefore, the sentiment of the text also changes accordingly. Negation can be expressed differently, making it somewhat challenging to detect. As a result, detecting negation is critical for Sentiment Analysis (SA) system development and improvement and will increase classifier accuracy, but it also poses a significant conceptual and technical challenge. This paper aims to survey and gather the most recent research related to detecting negation in SA. Many researchers have worked and performed methods, including algorithmic, machine, and deep learning approaches such as Decision Tree (DT), Support Vector Machines (SVM), K-Nearest Neighbor (KNN), Naive Bayesian (NB), Logistic Regression (LR), Artificial Neural Networks (ANNs), Recurrent Neural Networks (RNNs), Bidirectional Long ShortTerm Memory (BiLSTM), and other hybrid methods such as rule-based and machine learning, lexicon and machine learning, machine learning, and deep learning. It addresses and tries to identify the gaps in the current studies, laying the foundation for future studies in this field. [ABSTRACT FROM AUTHOR]

The article informs about cystic fibrosis (CF) and its impact on airway health, focusing on the role of CFTR mutations in causing abnormal ion transport and subsequent lung complications. Topic include the approval of elexacaftor/tezacaftor/ivacaftor (ETI), a CFTR modulator therapy, and its potential to improve lung function in patients with CF, highlighting a retrospective analysis to determine its effect on Pseudomonas aeruginosa (PA) acquisition and chronic infection rates.

Traditional methods of personality assessment, and survey-based research in general, cannot make inferences about new items that have not been surveyed previously. This limits the amount of information that can be obtained from a given survey. In this article, we tackle this problem by leveraging recent advances in statistical natural language processing. Specifically, we extract "embedding" representations of questionnaire items from deep neural networks, trained on large-scale English language data. These embeddings allow us to construct a high-dimensional space of items, in which linguistically similar items are located near each other. We combine item embeddings with machine learning algorithms to extrapolate participant ratings of personality items to completely new items that have not been rated by any participants. The accuracy of our approach is on par with incentivized human judges given an identical task, indicating that it predicts ratings of new personality items as accurately as people do. Our approach is also capable of identifying psychological constructs associated with questionnaire items and can accurately cluster items into their constructs based only on their language content. Overall, our results show how representations of linguistic personality descriptors obtained from deep language models can be used to model and predict a large variety of traits, scales, and constructs. In doing so, they showcase a new scalable and cost-effective method for psychological measurement. [ABSTRACT FROM AUTHOR]

Many studies project that, due to antibiotic misuse, phage therapy has been considered as one of the most promising alternatives for the treatment of human diseases infected by antibiotic-resistant bacteria. The identification of phage-host interactions (PHI) helps to explore the mechanisms by which bacteria respond to phages and provides new insights into effective therapeutic approaches. Computational models for predicting PHI are not only time/cost saving, but also more efficient and economical than traditional wet experiments. In this work, we proposed a deep learning based computational model named DWPREPHI to predict PHI through the combining DNA and protein sequence information. More specially, DWPREPHI first extracted information about the node properties of the interaction network by a natural language processing algorithm that initialized the node representations of the phage and the target bacterial host. The graph embedding algorithm, Deepwalk, was then used to extract link behavior information from the PHI network, and finally a deep neural network was applied to accurately detect interactions between phages and their bacterial hosts. On the drug-resistant bacteria dataset ESKAPE, DWPREPHI achieved a prediction accuracy of 92.25% and an AUC value of 0.9674 under the 5-fold cross-validation method, which was significantly better than other methods. In addition, three case studies were conducted for E. coli, Pseudomonas aeruginosa, and Salmonella enterica to further demonstrate the utility of the proposed model. Among the top 10 phages associated with these hosts, 7, 8, and 8 have been reported. These excellent experimental results suggested that the DWPREPHI model could provide reasonable candidates for sensitive bacteria for biological experiments in phage therapy. [ABSTRACT FROM AUTHOR]

Deep neural networks are known to be vulnerable to various types of adversarial attacks, especially word-level attacks, in the field of natural language processing. In recent years, various defense methods are proposed against word-level attacks; however, most of those defense methods only focus on synonyms substitution-based attacks, while word-level attacks are not based on synonym substitution. In this paper, we propose a textual adversarial defense method against word-level adversarial attacks via textual embedding based on the semantic associative field. More specifically, we analyze the reasons why humans can read and understand textual adversarial examples and observe two crucial points: (1) There must be a relation between the original word and the perturbed word or token. (2) Such a kind of relation enables humans to infer original words, while humans have the ability to associations. Motivated by this, we introduce the concept of semantic associative field and propose a new defense method by building a robust word embedding, that is, we calculate the word vector by exerting the related word vector to it with potential function and weighted embedding sampling for simulating the semantic influence between words in same semantic field. We conduct comprehensive experiments and demonstrate that the models using the proposed method can achieve higher accuracy than the baseline defense methods under various adversarial attacks or original testing sets. Moreover, the proposed method is more universal, while it is irrelevant to model structure and will not affect the efficiency of training. [ABSTRACT FROM AUTHOR]

The article focuses on the global burden of mental health disorders, especially in low- and middle-income countries (LAMICs), where stigma, delayed help-seeking, and a shortage of mental healthcare professionals contribute to suboptimal treatment and poor outcomes. It highlights the increasing mental health challenges exacerbated by the COVID-19 pandemic and explores the potential role of artificial intelligence (AI) as a solution to address the shortage of mental health professionals.

Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup can easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental learning, focusing on introducing FSCIL from two perspectives, while reviewing over 30 theoretical research studies and more than 20 applied research studies. From the theoretical perspective, we provide a novel categorization approach that divides the field into five subcategories, including traditional machine learning methods, meta learning-based methods, feature and feature space-based methods, replay-based methods, and dynamic network structure-based methods. We also evaluate the performance of recent theoretical research on benchmark datasets of FSCIL. From the application perspective, FSCIL has achieved impressive achievements in various fields of computer vision such as image classification, object detection, and image segmentation, as well as in natural language processing and graph. We summarize the important applications. Finally, we point out potential future research directions, including applications, problem setups, and theory development. Overall, this paper offers a comprehensive analysis of the latest advances in FSCIL from a methodological, performance, and application perspective. • In-depth survey on FSCIL methods, applications, performance. • Categorizing FSCIL into five subcategories for clear analysis. • Evaluating FSCIL research on benchmarks for strengths, weaknesses. • FSCIL applications in computer vision, NLP, and graph. [ABSTRACT FROM AUTHOR]

This document is a call for a new era of artificial intelligence (AI) in the connected world, specifically focusing on the field of Web Intelligence (WI). WI has evolved over the past 20 years, with three significant stages: WI 1.0 (Wisdom Web), WI 2.0 (Wisdom Web of Thing), and WI 3.0 (Wisdom Web of Everything). The document highlights several papers published in the Web Intelligence journal that explore different aspects of WI, including its impact on a better connected world, cross-cutting areas with brain informatics, biases in recommender systems, the influence of ChatGPT on information retrieval, the role of big data in the retail industry, order pairing mechanisms in real-time ride-sharing, data replication in cloud systems, and security risks in IoT-fog architecture. The document calls for more collaboration and communication among researchers in the WI field and emphasizes the importance of global interactions for co-intelligence and co-creation. [Extracted from the article]

In the digital age, social media has emerged as a significant platform, generating a vast amount of raw data daily. This data reflects the opinions of individuals from diverse backgrounds, races, cultures, and age groups, spanning a wide range of topics. Businesses can leverage this data to extract valuable insights, improve their services, and effectively reach a broader audience based on users' expressed opinions on social media platforms. To harness the potential of this extensive and unstructured data, a deep understanding of Natural Language Processing (NLP) is crucial. Existing approaches for sentiment analysis (SA) often rely on word co-occurrence frequencies, which prove inefficient in practical scenarios. Identifying this research gap, this paper presents a framework for concept-level sentiment analysis, aiming to enhance the accuracy of sentiment analysis (SA). A comprehensive Urdu language dataset was constructed by collecting data from YouTube, consisting of various talks and reviews on topics such as movies, politics, and commercial products. The dataset was further enriched by incorporating language rules and Deep Neural Networks (DNN) to optimize polarity detection. For sentiment analysis, the proposed framework employs predefined rules to trigger sentiment flow from words to concepts, leveraging the dependency relations among different words in a sentence based on Urdu language grammatical rules. In cases where predefined patterns are not triggered, the framework seamlessly switches to its sub-symbolic counterpart, passing the data to the DNN for sentence classification. Experimental results demonstrate that the proposed framework surpasses state-of-the-art approaches, including LSTM, CNN, SVM, LR, and MLP, achieving an improvement of 6–7% on Urdu dataset. In conclusion, this research paper introduces a novel framework for concept-level sentiment analysis of Urdu language data sourced from social media platforms. By combining language rules and DNN, the proposed framework demonstrates superior performance compared to existing methodologies, showcasing its effectiveness in accurately analyzing sentiment in Urdu text data. [ABSTRACT FROM AUTHOR]

In the past few years, it has become increasingly evident that deep neural networks are not resilient enough to withstand adversarial perturbations in input data, leaving them vulnerable to attack. Various authors have proposed strong adversarial attacks for computer vision and Natural Language Processing (NLP) tasks. As a response, many defense mechanisms have also been proposed to prevent these networks from failing. The significance of defending neural networks against adversarial attacks lies in ensuring that the model's predictions remain unchanged even if the input data is perturbed. Several methods for adversarial defense in NLP have been proposed, catering to different NLP tasks such as text classification, named entity recognition, and natural language inference. Some of these methods not only defend neural networks against adversarial attacks but also act as a regularization mechanism during training, saving the model from overfitting. This survey aims to review the various methods proposed for adversarial defenses in NLP over the past few years by introducing a novel taxonomy. The survey also highlights the fragility of advanced deep neural networks in NLP and the challenges involved in defending them. [ABSTRACT FROM AUTHOR]

Context: Developing scalable methods for conversation analytics is essential for health care communication science and quality improvement. Purpose: To assess the feasibility of automating the identification of a conversational feature, Connectional Silence, which is associated with important patient outcomes. Methods: Using audio recordings from the Palliative Care Communication Research Initiative cohort study, we develop and test an automated measurement pipeline comprising three machine-learning (ML) tools—a random forest algorithm and a custom convolutional neural network that operate in parallel on audio recordings, and subsequently a natural language processing algorithm that uses brief excerpts of automated speech-to-text transcripts. Results: Our ML pipeline identified Connectional Silence with an overall sensitivity of 84% and specificity of 92%. For Emotional and Invitational subtypes, we observed sensitivities of 68% and 67%, and specificities of 95% and 97%, respectively. Conclusion: These findings support the capacity for coordinated and complementary ML methods to fully automate the identification of Connectional Silence in natural hospital-based clinical conversations. [ABSTRACT FROM AUTHOR]

In recent years, a multitude of datasets of human–human conversations has been released for the main purpose of training conversational agents based on data‐hungry artificial neural networks. In this paper, we argue that datasets of this sort represent a useful and underexplored source to validate, complement, and enhance cognitive studies on human behavior and language use. We present a method that leverages the recent development of powerful computational models to obtain the fine‐grained annotation required to apply metrics and techniques from Cognitive Science to large datasets. Previous work in Cognitive Science has investigated the question‐asking strategies of human participants by employing different variants of the so‐called 20‐question‐game setting and proposing several evaluation methods. In our work, we focus on GuessWhat, a task proposed within the Computer Vision and Natural Language Processing communities that is similar in structure to the 20‐question‐game setting. Crucially, the GuessWhat dataset contains tens of thousands of dialogues based on real‐world images, making it a suitable setting to investigate the question‐asking strategies of human players on a large scale and in a natural setting. Our results demonstrate the effectiveness of computational tools to automatically code how the hypothesis space changes throughout the dialogue in complex visual scenes. On the one hand, we confirm findings from previous work on smaller and more controlled settings. On the other hand, our analyses allow us to highlight the presence of "uninformative" questions (in terms of Expected Information Gain) at specific rounds of the dialogue. We hypothesize that these questions fulfill pragmatic constraints that are exploited by human players to solve visual tasks in complex scenes successfully. Our work illustrates a method that brings together efforts and findings from different disciplines to gain a better understanding of human question‐asking strategies on large‐scale datasets, while at the same time posing new questions about the development of conversational systems. [ABSTRACT FROM AUTHOR]

Neural network models enjoy success on language tasks related to Web documents, including news and Wikipedia articles. However, the characteristics of scientific publications pose specific challenges that have yet to be satisfactorily addressed: the discourse structure of scientific documents crucial in scholarly document processing (SDP) tasks, the interconnected nature of scientific documents, and their multimodal nature. We survey modern neural network learning methods that tackle these challenges: those that can model discourse structure and their interconnectivity and use their multimodal nature. We also highlight efforts to collect large-scale datasets and tools developed to enable effective deep learning deployment for SDP. We conclude with a discussion on upcoming trends and recommend future directions for pursuing neural natural language processing approaches for SDP. [ABSTRACT FROM AUTHOR]

To what extent can neural network models learn generalizations about language structure, and how do we find out what they have learned? We explore these questions by training neural models for a range of natural language processing tasks on a massively multilingual dataset of Bible translations in 1,295 languages. The learned language representations are then compared to existing typological databases as well as to a novel set of quantitative syntactic and morphological features obtained through annotation projection. We conclude that some generalizations are surprisingly close to traditional features from linguistic typology, but that most of our models, as well as those of previous work, do not appear to have made linguistically meaningful generalizations. Careful attention to details in the evaluation turns out to be essential to avoid false positives. Furthermore, to encourage continued work in this field, we release several resources covering most or all of the languages in our data: (1) multiple sets of language representations, (2) multilingual word embeddings, (3) projected and predicted syntactic and morphological features, (4) software to provide linguistically sound evaluations of language representations. [ABSTRACT FROM AUTHOR]

Artificial neural networks (ANNs) are a type of machine learning model that are designed to mimic the structure and function of biological neurons. They are particularly well-suited for tasks such as image and speech recognition, natural language processing, and prediction tasks. The success of an ANN in modeling a particular dataset depends on factors such as the size and quality of the dataset, the complexity of the model, and the choice of training algorithms. High representation rate of a system in the data set can improve the performance of the ANN model. The study we described is focused on using artificial neural networks (ANNs) to model temperature-dependent photoluminescence (PL) characterization of GaN epilayers grown on patterned sapphire substrates (PSS) using the metalorganic chemical vapor deposition (MOCVD) technique. The ANN model is trained using temperature and wavelength as input parameters and intensity as the output parameter, with the goal of accurately predicting the PL intensity of the GaN epilayer as a function of temperature and wavelength. The model is trained using a large set of experimental data and then tested using data that was not presented to the model during training. The results of the study suggest that ANN modeling methodology is an effective and accurate way of modeling temperature-dependent PL of GaN epilayers grown on PSS. The results of the study suggest that ANN modeling methodology can be used to accurately predict the temperature-dependent PL of GaN epilayers grown on PSS. This means that it may be possible to reduce the number of required experimental measurements by using the ANN model to predict PL intensity at different temperatures, based on a smaller set of experimental measurements. This could potentially save time and resources, while still obtaining accurate information about the optical behavior of GaN-based materials at different temperatures. [ABSTRACT FROM AUTHOR]