Search

The process of retrieving suitable papers which are related to an interesting topic while doing research normally takes a lot of time and effort. Citation recommendation is frequently used to solve this problem by automatically suggesting a list of candidate papers that should match with the user's references or topics of interest. Applying the recent research results of deep learning in multiple disciplines, the performance of citation recommendation systems has been significantly improved with the facilitation of powerful deep neural data analysis and representation learning techniques. However, most of the recent Neural Citation Network (NCN) model-based techniques still encounter limitations related to the capability of integrating auxiliary information to assist the citation contextual learning process. Thus, in this paper, we propose a novel context-aware NCN-based model with the extra textual data integration and Bidirectional Encoder Representations from Transformers (BERT) model to improve the performance of the citation recommendation task. To do this, an extensive deep neural auto-encoding mechanism with a self-attention-based mechanism is utilized in our proposed model to flexibly learn both associated textual and citation contextual data in the given dataset. These enriched citation contextual information representations are then utilized to improve the performance of the citation recommendation task as the end-to-end neural learning process. Extensive experiments in the standard arXiv dataset show the effectiveness and good performance of the proposed model. [ABSTRACT FROM AUTHOR]

The transformer model is a famous natural language processing model proposed by Google in 2017. Now, with the extensive development of deep learning, many natural language processing tasks can be solved by deep learning methods. After the BERT model was proposed, many pre-trained models such as the XLNet model, the RoBERTa model, and the ALBERT model were also proposed in the research community. These models perform very well in various natural language processing tasks. In this paper, we describe and compare these well-known models. In addition, we also apply several types of existing and well-known models which are the BERT model, the XLNet model, the RoBERTa model, the GPT2 model, and the ALBERT model to different existing and well-known natural language processing tasks, and analyze each model based on their performance. There are a few papers that comprehensively compare various transformer models. In our paper, we use six types of well-known tasks, such as sentiment analysis, question answering, text generation, text summarization, name entity recognition, and topic modeling tasks to compare the performance of various transformer models. In addition, using the existing models, we also propose ensemble learning models for the different natural language processing tasks. The results show that our ensemble learning models perform better than a single classifier on specific tasks. [ABSTRACT FROM AUTHOR]

Copyright of Frontiers of Information Technology & Electronic Engineering is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the context of Industry 4.0, the development of intelligent manufacturing gave birth to a new generation of industrial systems which is expected to be operative under a high level of functionality, reliability, and resilience, to meet the high-quality engineering service and maintenance demands from the systems operators and end-users. Furthermore, experiments are conducted on a public wafer yield prediction dataset collected from an actual wafer manufacturer. The paper by Lu et al. entitled "Fault diagnosis of the industrial robot based on dual-module attention convolutional neural network" investigates a data-driven method for the fault diagnosis of industrial robot reducers, that is, a dual-module attention convolutional neural network (DMA-CNN). [Extracted from the article]

Deep learning has seen significant growth recently and is now applied to a wide range of conventional use cases, including graphs. Graph data provides relational information between elements and is a standard data format for various machine learning and deep learning tasks. Models that can learn from such inputs are essential for working with graph data effectively. This paper identifies nodes and edges within specific applications, such as text, entities, and relations, to create graph structures. Different applications may require various graph neural network (GNN) models. GNNs facilitate the exchange of information between nodes in a graph, enabling them to understand dependencies within the nodes and edges. The paper delves into specific GNN models like graph convolution networks (GCNs), GraphSAGE, and graph attention networks (GATs), which are widely used in various applications today. It also discusses the message-passing mechanism employed by GNN models and examines the strengths and limitations of these models in different domains. Furthermore, the paper explores the diverse applications of GNNs, the datasets commonly used with them, and the Python libraries that support GNN models. It offers an extensive overview of the landscape of GNN research and its practical implementations. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive review of deep learning applications in the video game industry, focusing on how these techniques can be utilized in game development, experience, and operation. As relying on computation techniques, the game world can be viewed as an integration of various complex data. This examines the use of deep learning in processing various types of game data. The paper classifies the game data into asset data, interaction data, and player data, according to their utilization in game development, experience, and operation, respectively. Specifically, this paper discusses deep learning applications in generating asset data such as object images, 3D scenes, avatar models, and facial animations; enhancing interaction data through improved text-based conversations and decision-making behaviors; and analyzing player data for cheat detection and match-making purposes. Although this review may not cover all existing applications of deep learning, it aims to provide a thorough presentation of the current state of deep learning in the gaming industry and its potential to revolutionize game production by reducing costs and improving the overall player experience. [ABSTRACT FROM AUTHOR]

In the globalization trend, China's cultural heritage is in danger of gradually disappearing. The protection and inheritance of these precious cultural resources has become a critical task. This paper focuses on the Miao batik culture in Guizhou Province, China, and explores the application of knowledge graphs, natural language processing, and deep learning techniques in the promotion and protection of batik culture. We propose a dual-channel mechanism that integrates semantic and visual information, aiming to connect batik pattern features with cultural connotations. First, we use natural language processing techniques to automatically extract batik-related entities and relationships from the literature, and construct and visualize a structured batik pattern knowledge graph. Based on this knowledge graph, users can textually search and understand the images, meanings, taboos, and other cultural information of specific patterns. Second, for the batik pattern classification, we propose an improved ResNet34 model. By embedding average pooling and convolutional operations into the residual blocks and introducing long-range residual connections, the classification performance is enhanced. By inputting pattern images into this model, their categories can be accurately identified, and then the underlying cultural connotations can be understood. Experimental results show that our model outperforms other mainstream models in evaluation metrics such as accuracy, precision, recall, and F1-score, achieving 94.46%, 94.47%, 93.62%, and 93.8%, respectively. This research provides new ideas for the digital protection of batik culture and demonstrates the great potential of artificial intelligence technology in cultural heritage protection. [ABSTRACT FROM AUTHOR]

Mental health monitoring of seafarers is an important part of achieving normal development of the ocean shipping industry. In this paper, a dual subjective–objective testing scheme is proposed to achieve a more effective and intelligent assessment of seafarers' mental health status. Firstly, a new seafarers' mental health test scale (SMHT) is revised based on fuzzy factor analysis and the test data of 283 marine practitioners are analyzed using SPSS v24 software; secondly, this paper proposes an intelligent framework module for immersive subjective emotion extraction based on natural language processing, namely semantic summary extraction (SSE), speech emotion extraction (SEE), using hybrid scoring mechanism to obtain semantic and emotion matching values and assist the seafarer mental health scale to obtain the final correction score. The results showed that the assessment results of the SMHT scale exhibited good reliability (Cronbach's alpha of 0.852 ∈ (0.80 - 0.90) and retest reliability R of 0.873 ∈ (0.85 - 0.90) ) and scale association validity (for SCL-90, ( r = 0.468 - 0.841) > 0.45 ). In addition, the calibration rate of the subject-object dual test method was improved by approximately 12.05% compared to the traditional mental health scale. Finally, the advantages and disadvantages of this solution were compared with mental health testing techniques such as CAT, machine learning, SCL-90, and fMRI, and the method demonstrated more accurate psychological testing results, providing a simple and intelligent solution for standardized psychological testing of seafarers. [ABSTRACT FROM AUTHOR]

Extracting text from an image using a Visual Question Answering (VQA) system is an application at the intersection of computer vision and Natural Language Processing (NLP) to help blind people better view and comprehend textual information within the image. A VQA system takes a natural language question and an image as its input and then targets different areas of the image to extract an answer from the text within the image. To detect and recognize text in the image, the Look-Read-Reason-Answer (LoRRA), DCD_ZJU and Multimodal Multi-Copy Mesh methods, classify image features by using an Optical-Character-Recognition (OCR) module. Due to the curvature, distortion, background noise, and different fonts, understanding the text in natural scenes is a challenge as OCR techniques are mainly designed to detect textual features from within clean image files. In this paper, we introduce a novel architecture that rectifies perspective and curved text in an image using Thin-Plate-Spline (TPS) transformations and detects objects through the utilization of You-Only-Look-Once networks to extract answers. The results of our experiments show that our proposed VQA method reaches higher accuracy levels on the OpenImage dataset as compared with the existing methods. [ABSTRACT FROM AUTHOR]

Self-supervision learning (SSL) has shown exceptionally promising results in natural language processing and, more recently, in image classification and recognition. Recent research works have demonstrated SSL's benefits on large unlabeled datasets. However, relatively little investigation has been done into how well it works with smaller datasets. Typically, this challenge entails training a model on a very small quantity of data and then evaluating the model on out-of-distribution data. Few-shot image classification aims to classify classes that haven't been seen before using a limited number of training examples. Recent few-shot learning research focuses on developing good representation models that can quickly adapt to test tasks. In this paper, we investigate the role of self-supervision in the context of few-shot learning. We devised a model that improves the network's representation learning by employing a self-supervised auxiliary task that is based on composite rotation. We propose a composite rotation-based auxiliary task that rotates the image on two levels: inner and outer, and assigns one of 16 rotation classes to the modified image. Then, we further trained our model, which enables us to capture the robust learnable features that assist in focusing on better visual details of an object present in the given image. We find that the network is able to learn to extract more generalized and discriminative features, which in turn helps to enhance the effectiveness of its few-shot classification. This approach significantly outperforms the state-of-the-art on several public benchmarks. In addition, we demonstrated empirically that models trained using the proposed approach perform better than the baseline model even when the query examples in the episode are not aligned with the support examples. Extensive ablation experiments are performed to validate the various components of our approach. We also investigate our strategy's impact on the network's ability to discriminate visual features. Graphical Abstract [ABSTRACT FROM AUTHOR]

Short text matching is a fundamental technique of natural language processing. It plays an important role in information retrieval, question answering and paraphrase identification, etc. However, due to the lack of available data after Chinese short text word segmentation, we need to take full advantage of the existing text information. In our paper, we propose a sentence matching model with multiway semantic interaction based on multi-granularity semantic embedding(MSIM) to dispose of the problem of Chinese short text matching. First, each sentence pair is represented as multi-granularity embedding: character embedding based on one hot vector, and word embedding obtained from the pre-trained model. In addition, we add the attention mechanism after the character embedding to weight the characters. In order to capture sufficient semantic features, we process short sentence pairs in three ways. We not only match each time step of the two encoded sentences and perform average pooling and maximum pooling operations, but also make deep interaction between each time step representation with attention representation. Finally, we employ BiLSTM to aggregate matching results into a fixed-length matching vector, with the decision made through a fully connected layer. Our method is evaluated on the Chinese datasets CCKS and ATEC. Experimental results demonstrate that the method in our paper takes full advantage of Chinese short text information, outperforming other methods. [ABSTRACT FROM AUTHOR]

The question answering system is frequently applied in the area of natural language processing (NLP) because of the wide variety of applications. It consists of answering questions using natural language. The problem is, in general, solved by employing a dataset that consists of an input text, a query, and the text segment or span from the input text that provides the question's answer. The ability to make human-level predictions from data has improved significantly thanks to deep learning models, particularly the Transformer architecture, which has been state-of-the-art in text-based models in recent years. This paper reviews studies related to the use of transformer models in the implementation of question-answering (QA) systems. The paper's first focus is on the attention and transformer models. A brief description of the architectures is presented by classifying them into models based on encoders, decoders, and on both Encoder-Decoder. Following that, we examine the most recent research trends in textual QA datasets by highlighting the architecture of QA systems and categorizing them according to various criteria. We survey also a significant set of evaluation metrics that have been developed in order to evaluate the models' performance. Finally, we highlight solutions built to simplify the implementation of Transformer models. [ABSTRACT FROM AUTHOR]

While originally designed for natural language processing tasks, the self-attention mechanism has recently taken various computer vision areas by storm. However, the 2D nature of images brings three challenges for applying self-attention in computer vision: (1) treating images as 1D sequences neglects their 2D structures; (2) the quadratic complexity is too expensive for high-resolution images; (3) it only captures spatial adaptability but ignores channel adaptability. In this paper, we propose a novel linear attention named large kernel attention (LKA) to enable self-adaptive and long-range correlations in self-attention while avoiding its shortcomings. Furthermore, we present a neural network based on LKA, namely Visual Attention Network (VAN). While extremely simple, VAN achieves comparable results with similar size convolutional neural networks (CNNs) and vision transformers (ViTs) in various tasks, including image classification, object detection, semantic segmentation, panoptic segmentation, pose estimation, etc. For example, VAN-B6 achieves 87.8% accuracy on ImageNet benchmark, and sets new state-of-the-art performance (58.2 PQ) for panoptic segmentation. Besides, VAN-B2 surpasses Swin-T 4 mIoU (50.1 vs. 46.1) for semantic segmentation on ADE20K benchmark, 2.6 AP (48.8 vs. 46.2) for object detection on COCO dataset. It provides a novel method and a simple yet strong baseline for the community. The code is available at https://github.com/Visual-Attention-Network. [ABSTRACT FROM AUTHOR]

Due to the rapid expansion of online platforms, review-based recommender systems have become the standard way to determine consumers' preferences for various goods. The purpose of this study is to address three significant issues with review-based approaches. First, these techniques suffer from a class-imbalance problem, where rating levels with lower proportions are partially neglected. As a result, their rating performance for such rare levels is unsatisfactory. To address this drawback, this paper proposes a flexible dual-optimizer network that enhances performance by utilizing classification and regression losses in the first attempt to solve this problem. Second, the typical review-based approach of preprocessing review texts with word embeddings will result in inadequate contextual information extraction. Thus, the bidirectional encoder representations from the transformers (BERT) method is introduced as the preprocessing technique to fully extract semantic information. Third, existing techniques make recommendations without considering that the user preferences may change over time. We suggest a time-varying feature extraction scheme that includes a multiscale convolutional neural network and bidirectional long short-term memory to solve this problem. We then develop an interactive component that summarizes the contextual information associated with the user-item pairings. The efficiency of the proposed time-varying attention with dual-optimizer (TADO) model is demonstrated through extensive tests on 12 benchmark datasets derived from Amazon Product Reviews. Regarding the existing state-of-the-art techniques, the proposed model attains a substantial performance increase of 22.69%, 8.75%, and 17.35%, respectively, over the Aspect-aware Latent Factor Model (ALFM), Multi-Pointer Co-Attention Networks for recommendation (MPCN), and Aspect-based Neural Recommender (ANR). Moreover, an ablation study demonstrates the importance of jointly employing the components developed for TADO. [ABSTRACT FROM AUTHOR]

Short text information has considerable commercial value and immeasurable social value. Natural language processing and short text sentiment analysis technology can organize and analyze short text information on the Internet. Natural language processing tasks such as sentiment classification have achieved satisfactory performance under a supervised learning framework. However, traditional supervised learning relies on large-scale and high-quality manual labels and obtaining high-quality label data costs a lot. Therefore, the strong dependence on label data hinders the application of the deep learning model to a large extent, which is the bottleneck of supervised learning. At the same time, short text datasets such as product reviews have an imbalance in the distribution of data samples. To solve the above problems, this paper proposes a method to predict label data according to semi-supervised learning mode and implements the MixMatchNL data enhancement method. Meanwhile, the Bert pre-training model is updated. The cross-entropy loss function in the model is improved to the Focal Loss function to alleviate the data imbalance in short text datasets. Experimental results based on public datasets indicate the proposed model has improved the accuracy of short text sentiment recognition compared with the previous update and other state-of-the-art models. [ABSTRACT FROM AUTHOR]

BERT, a pre-trained language model on the large-scale corpus, has made breakthrough progress in NLP tasks. However, the experimental data shows that the BERT model's application effect in Chinese tasks is not ideal. The reason is that we believe that only character-level embedding can be obtained through BERT. However, a single Chinese character often cannot express their comprehensive meaning. To improve the model's ability to understand phrase-level semantic information, this paper proposes an enhanced BERT based on the average pooling(AP-BERT). Our model uses an average pooling layer to act on token embedding and reconstructs the model's input embedding, which can effectively improve BERT's application effect in Chinese natural language processing. Experimental data show that our proposed method has been enhanced in the four tasks of Chinese text classification, named entity recognition, reading comprehension, and summary generation. This method can not only improve the application effect of the BERT model in Chinese tasks but also can be well applied to other pre-trained language models. [ABSTRACT FROM AUTHOR]

In document layout analysis, both computer vision based and natural language processing based methods are employed individually or integrated to enrich the feature information resources and to enforce object detection. To simultaneously leverage visual and textual modalities, this paper proposes a hierarchical multimodal (HiM) network to aggregate representative features from multi-source inputs with the introduction of complementary semantics and non-local context dependencies across grained scales. Different channel and spatial attention mechanisms are adapted to different modalities. The visual modality is based on conventional convolution network, while the textual modality focuses on embedding hierarchical textual vectors and positioning. The feature representations from multiple modalities are then integrated adaptively in feature pyramid network for subsequent region proposal processing. We have made database adaptation on PubLayNet, including inserting semi-structure elements and extending ground truth annotations by parsing PDF pages. On three popular benchmarks, including Article Regions, PubLayNet and DocBank, extensive experiments are carried out to verify the effectiveness and adaptability of HiM. [ABSTRACT FROM AUTHOR]

The aim of this paper is to analyze the Arab peoples reactions and attitudes towards the Russo-Ukraine War through the social media of posted tweets, as a fast means to express opinions. We scrapped over 3 million tweets using some keywords that are related to the war and performed sentiment, emotion, and partiality analyses. For sentiment analysis, we employed a voting technique of several pre-trained Arabic language foundational models. For emotion analysis, we utilized a pre-constructed emotion lexicon. The partiality is analyzed through classifying tweets as being 'Pro-Russia', 'Pro-Ukraine', or 'Neither'; and it indicates the bias or empathy towards either of the conflicting parties. This was achieved by constructing a weighted lexicon of n-grams related to either side. We found that the majority of the tweets carried 'Negative' sentiment. Emotions were not that obvious with a lot of tweets carrying 'Mixed Feelings'. The more decisive tweets conveyed either 'Joy' or 'Anger' emotions. This may be attributed to celebrating victory ('Joy') or complaining from destruction ('Anger'). Finally, for partiality analysis, the amount of tweets classified as being 'Pro-Ukraine' was slightly greater than Pro-Russia' at the beginning of the war (specifically from Feb 2022 till April 2022) then slowly began to decrease until they nearly converged at the start of June 2022 with a shift happening in the empathy towards Russia in August 2022. Our Interpretation for that is with the initial Russian fierce and surprise attack at the beginning and the amount of refugees who escaped to neighboring countries, Ukraine gained much empathy. However, by April 2022, Russian intensity has been decreased and with heavy sanctions the U.S. and West have applied on Russia, Russia has begun to gain such empathy with decrease on the Ukrainian side. [ABSTRACT FROM AUTHOR]

Many attempts have been made to overcome the challenges of automating textual emotion detection using different traditional deep learning models such as LSTM, GRU, and BiLSTM. But the problem with these models is that they need large datasets, massive computing resources, and a lot of time to train. Also, they are prone to forgetting and cannot perform well when applied to small datasets. In this paper, we aim to demonstrate the capability of transfer learning techniques to capture the better contextual meaning of the text and as a result better detection of the emotion represented in the text, even without a large amount of data and training time. To do this, we conduct an experiment utilizing a pre-trained model called EmotionalBERT, which is based on bidirectional encoder representations from transformers (BERT), and we compare its performance to RNN-based models on two benchmark datasets, with a focus on the amount of training data and how it affects the models' performance. [ABSTRACT FROM AUTHOR]

Sensitive data identification represents the initial and crucial step in safeguarding sensitive information. With the ongoing evolution of the industrial internet, including its interconnectivity across various sectors like the electric power industry, the potential for sensitive data to traverse different domains increases, thereby altering the composition of sensitive data. Consequently, traditional approaches reliant on sensitive vocabularies struggle to adequately address the challenges posed by identifying sensitive data in the era of information abundance. Drawing inspiration from advancements in natural language processing within the realm of deep learning, we propose a transferable Sensitive Data Identification method based on Product Quantization, named PQ-SDI. This innovative approach harnesses both the composition and contextual cues within textual data to accurately pinpoint sensitive information within the context of Mobile Edge Computing (MEC). Notably, PQ-SDI exhibits proficiency not only within a singular domain but also demonstrates adaptability to new domains following training on heterogeneous datasets. Moreover, the method autonomously identifies sensitive data throughout the entire process, eliminating the necessity for human upkeep of sensitive vocabularies. Extensive experimentation with the PQ-SDI model across four real-world datasets, resulting in performance improvements ranging from 2% to 5% over the baseline model and achieves an accuracy of up to 94.41%. In cross-domain trials, PQ-SDI achieved comparable accuracy to training and identification within the same domain. Furthermore, our experiments showcased the product quantization technique significantly reduces the parameter size by tens of times for the subsequent sensitive data identification phase, particularly beneficial for resource-constrained environments characteristic of MEC scenarios. This inherent advantage not only bolsters sensitive data protection but also mitigates the risk of data leakage during transmission, thus enhancing overall security measures in MEC environments. [ABSTRACT FROM AUTHOR]

There has been a steep increase in the use of machine learning for various healthcare applications like the diagnosis of diseases, drug discovery, medical image analysis, etc. Machine learning solutions are proven to be more efficient and less time-consuming than conventional approaches. In this paper, we leverage the advantages of machine learning models to enable a humanoid robot to assist mental health patients. Facial expression and human voice are some of the most demonstrative ways to analyze human emotions, especially for the mentally challenged. We carry out this assistance by constantly monitoring the patient's voice (or audio) and facial expressions to predict human emotions. To implement the model of audio monitoring, we train three different machine learning and deep learning models to compare and choose the better model. Similarly, for facial recognition, we train a deep learning model using a specific dataset to predict facial expressions from the video captured in real time. We then integrate the better-performing machine learning models into a web interface for demonstration. [ABSTRACT FROM AUTHOR]

Social media platforms like Facebook, YouTube, and Twitter are banking on developing machine learning models to help stop the spread of hateful speech on their platforms. The idea is that machine learning models that utilize natural language processing will detect hate speech faster and better than people can. Despite numerous progress has been made for resource reach language, only a few attempts have been made for Ethiopian Languages such as Afaan Oromo. This paper examines the viability of deep learning models for Afaan Oromo hate speech recognition. Toward this, the biggest dataset of hate speech was collected and annotated by the language experts. Variations of profound deep learning models such as CNN, LSTMs, BiLSTMs, LSTM, GRU, and CNN-LSTM are examined to evaluate their viability in identifying Afaan Oromo Hate speeches. The result uncovers that the model dependent on CNN and Bi-LSTM outperforms all the other investigated models with an average F1-score of 87%. [ABSTRACT FROM AUTHOR]

Nowadays, the internet and social media ease the process of reviewing products. Consumers expose their thoughts, opinions, and experiences about products and services on various forums, websites, and mobile apps. In effect, internet reviews become a decision-maker for many people before getting their desired goods. Actually, text sentiment analysis consists of extracting insights and sentiments from social texts and consumers' reviews. Hence, various organizations conduct this analysis in order to better understand the attitude as well as the feedback of their customers toward their products. Besides, many scientific researchers are also interested in the analysis of customers' reviews by labeling them into a set of sentiments using some text classification algorithms. The following paper provides a convolutional neural network (CNN) model to classify text reviews' sentiments as negative or positive. Also, we make a comparative analysis using our proposed CNN model and several models' representations of word embedding to get the most efficient model. The experiments are implemented on the Amazon reviews dataset, and the diverse model designs have achieved appropriate performances. Reached results discern the importance of including stop-word in sentiment analysis tasks, in fact, the stop words elimination can provoke an inaccurate prediction of sentiments. Practically, using stop words with the CNN model has improved the accuracy result by 2% opposing the CNN model that has ignored them. Furthermore, we procure that the employment of a random initialization approach provides better performance than supervised and embedding model vectors on large-scale datasets. Effectively, training the representation of word embedding allows the model to learn better features in less computation time. Moreover, our CNN model showed better performance than the baseline machine learning and deep learning methods and improved the accuracy of the CNN to 90% on the Amazon reviews dataset. [ABSTRACT FROM AUTHOR]

The prolonged COVID-19 pandemic has tied up significant medical resources, and its management poses a challenge for the public health care decision making. Accurate predictions of the hospitalizations are crucial for the decision makers to make informed decision for the medical resource allocation. This paper proposes a method named County Augmented Transformer (CAT). To generate accurate predictions of four-week-ahead COVID-19 related hospitalizations for every states in the United States. Inspired by the modern deep learning techniques, our method is based on a self-attention model (known as the transformer model) that is actively used in Natural Language Processing. Our transformer based model can capture both short-term and long-term dependencies within the time series while enjoying computational efficiency. Our model is a data based approach that utilizes the publicly available information including the COVID-19 related number of confirmed cases, deaths, hospitalizations data, and the household median income data. Our numerical experiments demonstrate the strength and the usability of our model as a potential tool for assisting the medical resources allocation. [ABSTRACT FROM AUTHOR]

Sentiment lexicon is a reliable resource in computing sentiment classification. However, a general purpose lexicon alone is not sufficient, since text sentiment classification is perceived as a context-dependent task in the literature. On the contrary, we observe that many people tend to imitate others while writing reviews. As such, the subject of all the public opinion towards an entity ends up as an imbalanced corpus. In this paper, we intend to induce a context-based lexicon as a resource to explore imbalanced text sentiment classification. This method addresses the above mentioned two critical problems in text sentiment classification. First, it identifies subjective words relative to the context and computes the weight scores for subjective terms and full review. Also, in recent years, the application of RNNs to a variety of problems has been incredible, especially in natural language processing tasks. Thus, we take the advantages of the context-based lexicon as well as a bidirectional LSTM to handle text sentiment classification. Second, it deals imbalanced data by deploying a text based oversampling method for creating new synthetic text samples. The reason behind using a text based oversampling method is to make use of semantics of the information while creating new text samples. Experimental results prove that leveraging sentiment lexicon relative to the context and application of Bidiricetional LSTM with text based oversampling is useful in imbalanced text sentiment classification and in achieving state-of-the-art results over deep neural learning model baselines. [ABSTRACT FROM AUTHOR]

Heart disease remains the major cause of death, despite recent improvements in prediction and prevention. Risk factor identification is the main step in diagnosing and preventing heart disease. Automatically detecting risk factors for heart disease in clinical notes can help with disease progression modeling and clinical decision-making. Many studies have attempted to detect risk factors for heart disease, but none have identified all risk factors. These studies have proposed hybrid systems that combine knowledge-driven and data-driven techniques, based on dictionaries, rules, and machine learning methods that require significant human effort. The National Center for Informatics for Integrating Biology and Beyond (i2b2) proposed a clinical natural language processing (NLP) challenge in 2014, with a track (track2) focused on detecting risk factors for heart disease risk factors in clinical notes over time. Clinical narratives provide a wealth of information that can be extracted using NLP and Deep Learning techniques. The objective of this paper is to improve on previous work in this area as part of the 2014 i2b2 challenge by identifying tags and attributes relevant to disease diagnosis, risk factors, and medications by providing advanced techniques of using stacked word embeddings. The i2b2 heart disease risk factors challenge dataset has shown significant improvement by using the approach of stacking embeddings, which combines various embeddings. Our model achieved an F1 score of 93.66% by using BERT and character embeddings (CHARACTER-BERT Embedding) stacking. The proposed model has significant results compared to all other models and systems that we developed for the 2014 i2b2 challenge. [ABSTRACT FROM AUTHOR]

This paper proposes an automatic Chinese text categorization method for solving the emergency event report classification problem. Since the bidirectional encoder representations from transformers (BERT) has achieved great success in the natural language processing domain, it is employed to derive emergency text features in this study. To overcome the data imbalance problem in the distribution of emergency event categories, a novel loss function is proposed to improve the performance of the BERT-based model. Meanwhile, in order to avoid the negative impacts of the extreme learning rate, the Adabound optimization algorithm that achieves a gradual smooth transition from Adam optimizer to stochastic gradient descent optimizer is employed to learn the parameters of the model. The feasibility and competitiveness of the proposed method are validated on both imbalanced and balanced datasets. Furthermore, the generic BERT, BERT ensemble LSTM-BERT (BERT-LB), Attention-based BiLSTM fused CNN with gating mechanism (ABLG-CNN), TextRCNN, Att-BLSTM, and DPCNN are used as benchmarks on these two datasets. Meanwhile, sampling methods, including random sampling, ADASYN, synthetic minority over-sampling techniques (SMOTE), and Borderline-SMOTE, are employed to verify the performance of the proposed loss function on the imbalance dataset. Compared with benchmarking methods, the proposed method has achieved the best performance in terms of accuracy, weighted average precision, weighted average recall, and weighted average F1 values. Therefore, it is promising to employ the proposed method for real applications in smart emergency management systems. [ABSTRACT FROM AUTHOR]

Cyberspace is a vast soapbox for people to post anything that they witness in their day-to-day lives. Social media content is mostly used for review, opinion, influence, or sentiment analysis. In this paper, we aim to extend sentiment and emotion analysis for detecting the stress of an individual based on the posts and comments shared by him/her on social networking platforms. We leverage large-scale datasets with tweets to accomplish sentiment analysis with the aid of machine learning algorithms and a deep learning model, BERT for sentiment classification. We also adopted Latent Dirichlet Allocation which is an unsupervised machine learning method for scanning a group of documents, recognizing the word and phrase patterns within them, and gathering word groups and alike expressions that most precisely illustrate a set of documents. This helps us to predict which topic is linked to the textual data. With the aid of these models, we will be able to detect the emotion of users online. Further, these emotions can be used to analyze stress or depression. In conclusion, the ML models and a BERT model have a very good detection rate. This research is useful for the well-being of one's mental health. The results are evaluated using various metrics at the macro and micro levels and indicate that the trained model detects the status of emotions based on social interactions. [ABSTRACT FROM AUTHOR]

Image captioning spans the fields of computer vision and natural language processing. The image captioning task generalizes object detection where the descriptions are a single word. Recently, most research on image captioning has focused on deep learning techniques, especially Encoder-Decoder models with Convolutional Neural Network (CNN) feature extraction. However, few works have tried using object detection features to increase the quality of the generated captions. This paper presents an attention-based, Encoder-Decoder deep architecture that makes use of convolutional features extracted from a CNN model pre-trained on ImageNet (Xception), together with object features extracted from the YOLOv4 model, pre-trained on MS COCO. This paper also introduces a new positional encoding scheme for object features, the "importance factor". Our model was tested on the MS COCO and Flickr30k datasets, and the performance is compared to performance in similar works. Our new feature extraction scheme raises the CIDEr score by 15.04%. The code is available at: https://github.com/abdelhadie-almalla/image_captioning [ABSTRACT FROM AUTHOR]

The ubiquity of social media and their deep integration in the contemporary society has granted new ways to interact, exchange information, form groups, or earn money—all on a scale never seen before. Those possibilities paired with the widespread popularity contribute to the level of impact that social media display. Unfortunately, the benefits brought by them come at a cost. Social Media can be employed by various entities to spread disinformation—so called 'Fake News', either to make a profit or influence the behaviour of the society. To reduce the impact and spread of Fake News, a diverse array of countermeasures were devised. These include linguistic-based approaches, which often utilise Natural Language Processing (NLP) and Deep Learning (DL). However, as the latest advancements in the Artificial Intelligence (AI) domain show, the model's high performance is no longer enough. The explainability of the system's decision is equally crucial in real-life scenarios. Therefore, the objective of this paper is to present a novel explainability approach in BERT-based fake news detectors. This approach does not require extensive changes to the system and can be attached as an extension for operating detectors. For this purposes, two Explainable Artificial Intelligence (xAI) techniques, Local Interpretable Model-Agnostic Explanations (LIME) and Anchors, will be used and evaluated on fake news data, i.e., short pieces of text forming tweets or headlines. This focus of this paper is on the explainability approach for fake news detectors, as the detectors themselves were part of previous works of the authors. [ABSTRACT FROM AUTHOR]

Discourse parsing of scholarly documents is the premise and basis for standardizing the writing of scholarly documents, understanding their content, and quickly locating and extracting specific information from them. With the continuous emergence of a large number of scholarly documents, how to automatically analyze scholarly documents quickly and effectively has become a research hotspot. In this paper, we propose a hybrid model, which considers both section headers and body texts, to recognize generic sections in scholarly documents automatically. We conduct a comprehensive analysis of the semantic difference between short phrases and long narrative text chunks on the SectLabel dataset. The experimental results show that our model achieves 91.67% F 1 -value in the generic section recognization, which is better than the baseline. [ABSTRACT FROM AUTHOR]

Microelectronics production failure analysis is an important step in improving product quality and development. In fact, the understanding of the failure mechanisms and therefore the implementation of corrective actions on the cause of the failure depend on the results of this analysis. These analyses are saved under textual features format. Then such data need first to be preprocessed and vectorized (converted to numeric). Second, to overcome the curse of dimensionality caused by the vectorisation process, a dimension reduction is applied. A two-stage variable selection and feature extraction is used to reduce the high dimensionality of a feature space. We are first interested in studying the potential of using an unsupervised variable selection technique, the genetic algorithm, to identify the variables that best demonstrate discrimination in the separation and compactness of groups of textual data. The genetic algorithm uses a combination of the K-means or Gaussian Mixture Model clustering and validity indices as a fitness function for optimization. Such a function improves both compactness and class separation. The second work looks into the feasibility of applying a feature extraction technique. The adopted methodology is a Deep learning algorithm based on variational autoencoder (VAE) for latent space disentanglement and Gaussian Mixture Model for clustering of the latent space for cluster identification. The last objective of this paper is to propose a new methodology based on the combination between variational autoencoder (VAE) for the latent space disentanglement, and genetic algorithm (GA) to find, in an unsupervised way, the latent variables allowing the best discrimination of clusters of failure analysis data. This methodology is called VAE-GA. Experiments on textual datasets of failure analysis demonstrate the effectiveness of the VAE-GA proposed method which allows better discrimination of textual classes compared to the use of GA or VAE separately or the combination of PCA with GA (PCA-GA) or a simple Auto-encoders with GA (AE-GA). [ABSTRACT FROM AUTHOR]

Business process compliance is an essential part of business process management, which saves organizations from penalties caused by non-compliant processes. However, current researches on business process compliance mainly focus on checking using general constraint rules that have been formalized without in-depth analysis of related regulatory documents and mostly involve extensive human efforts. In this paper, we aim to propose an automatic and interpretable compliance checking approach for design-time business processes. By combining deep learning and a natural language processing approach based on rule templates, we extract semantic information from regulatory documents at different granularities for subsequent compliance checking. In addition, we match appropriate rules to the process model and detect the degree of violation of the business process from three control-flow perspectives. The effectiveness of this method is validated on two real-world datasets. [ABSTRACT FROM AUTHOR]

Semantic relatedness is one of the most significant aspects of natural language processing. It has been identified as a critical technology for developing intelligent systems like Siri, Microsoft Ice, Cortana, and Xiaoai. In 2014, SemEval ranked SR as the top task. While many existing studies have focused on analyzing the entailment of single phrases, advancements in deep learning have made it possible to analyze complete sentences or texts. While the natural parallelism of membrane computing has shown promise for data processing, harnessing this potential to advance semantic relatedness remains an open problem yet to be tackled. This paper proposes a novel Siamese Manhattan LSTM-SNP approach (SiMaLSTM-SNP) for the SR problem. The approach uses a collaborative Word2vec and 10-Layer Attention strategy to represent and extract sentence pairs and a Siamese LSTM-SNP structure to calculate the hidden states of sentences. The multi-head self-attention layer identifies text associations and redistributes hidden state weights. The last hidden state is extracted, and the relatedness score is calculated using the Manhattan distance. The experiments demonstrate that SiMaLSTM-SNP outperforms 17 classical SR baselines and 7 novel approaches on the standard datasets SICK and STS in terms of mean square error performance. This indicates that SiMaLSTM-SNP can accurately capture the semantic distinction between two sentences and effectively preserve their semantic information. [ABSTRACT FROM AUTHOR]

Rain impairs the performance of outdoor vision systems, such as automated driving systems and outdoor surveillance systems. Therefore, as an image preprocessing technique, image deraining has great potential for application. Defects of convolutional neural networks (small receptive field and non-adaptive to input content) limit the further improvement of deraining model performance. Recently, a novel neural network, transformer, has demonstrated impressive performance on natural language processing and vision tasks. However, using transformer for image deraining still has some issues: Although transformers have powerful long-range computing capabilities, it lacks the ability to model local features, which is critical for image deraining. In addition, transformer uses fixed-size patches to process images, which leads to pixels at the edges of the patches that cannot use the local features of neighboring pixels to restore rain-free images. In this paper, we propose a novel pyramid transformer for image deraining. To address the first issue, we design a residual-Dconv feed-forward network (RDFN), where depth-wise convolution improves the capability of modeling local features. To address the second issue, we introduce multi-resolution features into the transformer, which allows the transformer to obtain patches with different scales, thus enabling the boundary pixels to utilize local features. Furthermore, we propose a novel multi-scale fusion bridge (MSFB) to effectively integrate the extracted multi-scale features and capture the correlation between different scales. Extensive experiments on synthetic and real-world images demonstrate that the proposed deraining model achieves superior performance, especially the PSNR value achieves 47.55 dB on the SPA-Data dataset. We also further validate the effectiveness of the proposed model on subsequent high-level computer vision tasks. [ABSTRACT FROM AUTHOR]

Recent advancements in computer science and engineering have significantly boosted interest in and success rate of speech recognition systems. Sophisticated speech recognition systems are being developed for a variety of languages having wide-spread applications. Arabic being one of the most widely used languages in the world, augmented by religious importance, is a potential candidate in this domain, however rich vocabulary and multiple dialects open up new challenges for development of Arabic speech recognition processes. This paper focuses on development of an Arabic isolated speech recognition system for the vocabulary of Holy Quran using Mel frequency cepstral coefficients and deep neural networks. The proposed system is capable of identifying individual words from a recited verse with reasonable accuracy. It targets the 362 unique words of the first and the last 19 chapters of the Holy Quran and uses 14 hours of audio data to demonstrate the working prototype of the system. A user-friendly, web-based application has also been developed for transcription of recitations. The software is specified to make an effort in spreading the right way of recitation of the Holy Quran. The presented architecture can be extended to a complete automatic speech recognition system to help people practice recitations, mapping speech to transcription and then matching the transcribed utterance with standard templates of correct recitation to identify reciter's short comings. [ABSTRACT FROM AUTHOR]

Radiology reports contain a diverse and rich set of clinical abnormalities documented by radiologists during their interpretation of the images. Comprehensive semantic representations of radiological findings would enable a wide range of secondary use applications to support diagnosis, triage, outcomes prediction, and clinical research. In this paper, we present a new corpus of radiology reports annotated with clinical findings. Our annotation schema captures detailed representations of pathologic findings that are observable on imaging ("lesions") and other types of clinical problems ("medical problems"). The schema used an event-based representation to capture fine-grained details, including assertion, anatomy, characteristics, size, and count. Our gold standard corpus contained a total of 500 annotated computed tomography (CT) reports. We extracted triggers and argument entities using two state-of-the-art deep learning architectures, including BERT. We then predicted the linkages between trigger and argument entities (referred to as argument roles) using a BERT-based relation extraction model. We achieved the best extraction performance using a BERT model pre-trained on 3 million radiology reports from our institution: 90.9–93.4% F1 for finding triggers and 72.0–85.6% F1 for argument roles. To assess model generalizability, we used an external validation set randomly sampled from the MIMIC Chest X-ray (MIMIC-CXR) database. The extraction performance on this validation set was 95.6% for finding triggers and 79.1–89.7% for argument roles, demonstrating that the model generalized well to the cross-institutional data with a different imaging modality. We extracted the finding events from all the radiology reports in the MIMIC-CXR database and provided the extractions to the research community. [ABSTRACT FROM AUTHOR]

Machine learning is a branch of the field of artificial intelligence. Deep learning is a complex machine learning algorithm that has unique advantages in image recognition, speech recognition, natural language processing, and industrial process control. Deep learning has It is widely used in the field of wireless communication. Prediction of geological disasters (such as landslides) is currently a difficult problem. Because landslides are difficult to detect in the early stage, this paper proposes a GPS-based wireless communication continuous detection system and applies it to landslide deformation monitoring to achieve early treatment and prevention. This article introduces the GPS multi-antenna detection system based on deep learning wireless communication, and introduces the time series analysis method and its application. The test results show that the GPS multi-antenna detection system of the wireless communication network has great advantages in response time, with high accuracy and small error. The horizontal accuracy is controlled at 0–2 mm and the vertical accuracy is about 1 mm. The analysis method is simple and efficient, and can obtain good results for short-term deformation prediction. [ABSTRACT FROM AUTHOR]

Guest Advisory Editor: R. K. Shyamasundar is a Fellow of IEEE, a Fellow of ACM, Distinguished ACM Speaker, a Distinguished Alumnus of Indian Institute of Science. As the organizers of the SKM-2019 conference we hope that this special issue in Information Systems Frontiers will highlight the current trends in SKM and will inspire more research on the application of AI for SKM. We see an increasing trend of AI-based systems designed to attack other AI systems that are involved in generating advanced insights from the data (e.g., supervised deep learning systems). [Extracted from the article]

There is an exponential growth in textual content generation every day in today's world. In-app messaging such as Telegram and WhatsApp, social media websites such as Instagram and Facebook, e-commerce websites like Amazon, Google searches, news publishing websites, and a variety of additional sources are the possible suppliers. Every instant, all these sources produce massive amounts of text data. The interpretation of such data can help business owners analyze the social outlook of their product, brand, or service and take necessary steps. The development of a consumer review summarization model using Natural Language Processing (NLP) techniques and Long short-term memory (LSTM) to present summarized data and help businesses obtain substantial insights into their consumers' behavior and choices is the topic of this research. A hybrid approach for analyzing sentiments is presented in this paper. The process comprises pre-processing, feature extraction, and sentiment classification. Using NLP techniques, the pre-processing stage eliminates the undesirable data from input text reviews. For extracting the features effectively, a hybrid method comprising review-related features and aspect-related features has been introduced for constructing the distinctive hybrid feature vector corresponding to each review. The sentiment classification is performed using the deep learning classifier LSTM. We experimentally evaluated the proposed model using three different research datasets. The model achieves the average precision, average recall, and average F1-score of 94.46%, 91.63%, and 92.81%, respectively. [ABSTRACT FROM AUTHOR]

Cross-border trade barriers introduced by national authorities to protect local business and labor force cause substantial damage to international economical actors. Therefore, identifying such barriers beyond regulator's audit reporting is of paramount importance. This paper contributes towards this goal by proposing a novel approach that uses natural language processing and deep learning method for uncovering Finnish-Russian trade barriers in the fish industry from selected business discussion forums. Especially, the approach makes use i) a three-leg ontology for data collection, ii) a BERT architecture for mapping Onkivisit-Shaw-Kananen trade barrier ontology to negative polarity posts and, iii) a new reverse-engineering clustering approach to identify the causes of individual trade-barrier types. A comparison with official statistical reports has been carried out to identify the salient aspects of trade-barriers that hold regardless of the time difference. The findings reveal the dominance of the Time-length barrier type in the Finnish discussion forum dataset and import vs export tariff discrepancy and product requirement barrier types in the Russian forum dataset. The developed framework can serve as a tool to assist companies or regulators in providing business-related recommendations to overcome the detected trade barriers. [ABSTRACT FROM AUTHOR]

Macroevolution can be regarded as the result of evolutionary changes of synergistically acting genes. Unfortunately, the importance of these genes in macroevolution is difficult to assess and hence the identification of macroevolutionary key genes is a major challenge in evolutionary biology. In this study, we designed various word embedding libraries of natural language processing (NLP) considering the multiple mechanisms of evolutionary genomics. A novel method (IKGM) based on three types of attention mechanisms (domain attention, kmer attention and fused attention) were proposed to calculate the weights of different genes in macroevolution. Taking 34 species of diurnal butterflies and nocturnal moths in Lepidoptera as an example, we identified a few of key genes with high weights, which annotated to the functions of circadian rhythms, sensory organs, as well as behavioral habits etc. This study not only provides a novel method to identify the key genes of macroevolution at the genomic level, but also helps us to understand the microevolution mechanisms of diurnal butterflies and nocturnal moths in Lepidoptera. [ABSTRACT FROM AUTHOR]

Automated audio captioning is a cross-modal translation task that aims to generate natural language descriptions for given audio clips. This task has received increasing attention with the release of freely available datasets in recent years. The problem has been addressed predominantly with deep learning techniques. Numerous approaches have been proposed, such as investigating different neural network architectures, exploiting auxiliary information such as keywords or sentence information to guide caption generation, and employing different training strategies, which have greatly facilitated the development of this field. In this paper, we present a comprehensive review of the published contributions in automated audio captioning, from a variety of existing approaches to evaluation metrics and datasets. We also discuss open challenges and envisage possible future research directions. [ABSTRACT FROM AUTHOR]

An image caption generator produces syntactically and semantically correct sentences to narrate the scene of a natural image. A neural image caption (NIC) generator is a popular deep learning model for automatically generating image captions in plain English. The NIC generator combines a convolutional neural network (CNN) encoder and a long short-term memory (LSTM) decoder. This paper investigates the performance of different CNN encoders and recurrent neural network decoders for finding the best NIC generator model for image captioning. Besides, we test the image caption generators with four image inject models and with decoding strategies such as greedy search and beam search. We conducted experiments on the Flickr8k dataset and analyzed the results qualitatively and quantitatively. Our results validate the automated image caption generator with ResNet-101 encoder, and the LSTM/gated recurrent units decoder outperforms the popular neural image caption NIC generator in the presence of par-inject concatenate conditioning and beam search. For quantitative assessment, we used R O U G E L , C I D E r D , and B L E U n scores to compare the different models. [ABSTRACT FROM AUTHOR]

The main objective of multilingual sentiment analysis is to analyze reviews regardless of the original language in which they are written. Switching from one language to another is very common on social media platforms. Analyzing these multilingual reviews is a challenge since each language is different in terms of syntax, grammar, etc. This paper presents a new language-independent representation approach for sentiment analysis, SentiCode. Unlike previous work in multilingual sentiment analysis, the proposed approach does not rely on machine translation to bridge the gap between different languages. Instead, it exploits common features of languages, such as part-of-speech tags used in Universal Dependencies. Equally important, SentiCode enables sentiment analysis in multi-language and multi-domain environments simultaneously. Several experiments were conducted using machine/deep learning techniques to evaluate the performance of SentiCode in multilingual (English, French, German, Arabic, and Russian) and multi-domain environments. In addition, the vocabulary proposed by SentiCode and the effect of each token were evaluated by the ablation method. The results highlight the 70% accuracy of SentiCode, with the best trade-off between efficiency and computing time (training and testing) in a total of about 0.67 seconds, which is very convenient for real-time applications. [ABSTRACT FROM AUTHOR]

With advances in deep learning and natural language processing (NLP), the analysis of medical texts is becoming increasingly important. Nonetheless, despite the importance of processing medical texts, no research on Korean medical-specific language models has been conducted. The Korean medical text is highly difficult to analyze because of the agglutinative characteristics of the language, as well as the complex terminologies in the medical domain. To solve this problem, we collected a Korean medical corpus and used it to train the language models. In this paper, we present a Korean medical language model based on deep learning NLP. The model was trained using the pre-training framework of BERT for the medical context based on a state-of-the-art Korean language model. The pre-trained model showed increased accuracies of 0.147 and 0.148 for the masked language model with next sentence prediction. In the intrinsic evaluation, the next sentence prediction accuracy improved by 0.258, which is a remarkable enhancement. In addition, the extrinsic evaluation of Korean medical semantic textual similarity data showed a 0.046 increase in the Pearson correlation, and the evaluation for the Korean medical named entity recognition showed a 0.053 increase in the F1-score. [ABSTRACT FROM AUTHOR]

Coreference Resolution is an essential task for Natural Language Processing (NLP) application, which has a paramount impact on the performance of text summarization, machine translation, text classification, and recognizing textual entailment. Mention Detection (MD) is the core component of the coreference resolution task and is additionally a process of extraction of all possible mentions from the text. Mention is referred to as a textual representation of entities in the text, such as Name, Nominal, and Pronominal mentions. The mentions appear in the text using different representations but indicating the same entity. The performance of an MD module positively affects the performance of NLP tasks such as Coreference resolution, Relation Extraction, Information retrieval, Information extraction, etc. Incorrect identification of mentions in the text severely affects the efficiency of the coreference resolution task. This paper aims to provide a comprehensive overview for the state of the art of mention detection approaches, which is utilized in the coreference resolution task and explains the importance of MD in Coreference resolution. The subsisting approaches are classified based on the underlying techniques adopted by each approach in three categories: Rule-based mention detection, Statistics-based mention detection, and Deep learning-based mention detection. The performance of deep learning is improving as more data and more powerful computing resources become available. This study endeavors to provide a comparative analysis of various mention detection approaches and help the researchers to assimilate knowledge about the mention detection approaches from sundry aspects. [ABSTRACT FROM AUTHOR]

Due to the rapid growth of textual information on the web, analyzing users' opinions about particular products, events or services is now considered a crucial and challenging task that has changed sentiment analysis from an academic endeavor to an essential analytic tool in cognitive science and natural language understanding. Despite the remarkable success of deep learning models for textual sentiment classification, they are still confronted with some limitations. Convolutional neural network is one of the deep learning models that has been excelled at sentiment classification but tends to need a large amount of training data while it considers that all words in a sentence have equal contribution in the polarity of a sentence and its performance is highly dependent on its accompanying hyper-parameters. To overcome these issues, an Attention-Based Convolutional Neural Network with Transfer Learning (ACNN-TL) is proposed in this paper that not only tries to take advantage of both attention mechanism and transfer learning to boost the performance of sentiment classification but also language models, namely Word2Vec and BERT, are used as its the backbone to better express sentence semantics as word vector. We conducted our experiment on widely-studied sentiment classification datasets and according to the empirical results, not only the proposed ACNN-TL achieved comparable or even better classification results but also employing contextual representation and transfer learning yielded remarkable improvement in the classification accuracy. [ABSTRACT FROM AUTHOR]

Zigzag conversational patterns of contents in social media are often perceived as noisy or informal text. Unrestricted usage of vocabulary in social media communications complicates the processing of code-mixed text. This paper accentuates two major aspects of code mixed text: Offensive Language Identification and Sentiment Analysis for Malayalam–English code-mixed data set. The proffered framework addresses 3 key points apropos these tasks—dependencies among features created by embedding methods (Word2Vec and FastText), comparative analysis of deep learning algorithms (uni-/bi-directional models, hybrid models, and transformer approaches), relevance of selective translation and transliteration and hyper-parameter optimization—which ensued in F1-Scores (model's accuracy) of 0.76 for Forum for Information Retrieval Evaluation (FIRE) 2020 and 0.99 for European Chapter of the Association for Computational Linguistics (EACL) 2021 data sets. A detailed error analysis was also done to give meaningful insights. The submitted strategy turned in the best results among the benchmarked models dealing with Malayalam–English code-mixed messages and it serves as an important step towards societal good. [ABSTRACT FROM AUTHOR]

Sentiment analysis for user reviews has received substantial heed in recent years. There are many deep learning models for natural language processing (NLP) applications. Long-short term memory (LSTM) and Convolutional neural network (CNN) based models efficiently enhance sentiment accuracy. Aspect-level sentiment analysis involves aspect extraction, aspect categorization, and polarity classification. The aspect sentiments in the dataset are classified as positive, negative, and neutral, depending on the polarity score associated with the aspect emotions. Existing neural architectures combining LSTM and CNN employ only the implicit information from the dataset for sentiment classification. Alternatively, this paper highlights the integration of explicit knowledge from the external database (RecogNet) with the implicit information of the LSTM model to improvise the sentiment accuracy. Incorporating sentic and semantic clues from the RecogNet knowledge base to the LSTM increases aspect extraction and categorization efficiency. Furthermore, we implemented CNN with target and position attention mechanisms over the RecogNet-LSTM layer to further enhance the classification accuracy. Finally, the model evaluations are performed using five online datasets related to the restaurants, laptops, and locations. Among LSTM based hybrid models, our RecogNet-LSTM+CNN model with attention mechanism showed superior performance in aspect categorization and opinion classification. [ABSTRACT FROM AUTHOR]