Search

The mining of Tweets to develop datasets on recent issues, global challenges, pandemics, virus outbreaks, emerging technologies, and trending matters has been of significant interest to the scientific community in the recent past, as such datasets serve as a rich data resource for the investigation of different research questions. Furthermore, the virus outbreaks of the past, such as COVID-19, Ebola, Zika virus, and flu, just to name a few, were associated with various works related to the analysis of the multimodal components of Tweets to infer the different characteristics of conversations on Twitter related to these respective outbreaks. The ongoing outbreak of the monkeypox virus, declared a Global Public Health Emergency (GPHE) by the World Health Organization (WHO), has resulted in a surge of conversations about this outbreak on Twitter, which is resulting in the generation of tremendous amounts of Big Data. There has been no prior work in this field thus far that has focused on mining such conversations to develop a Twitter dataset. Furthermore, no prior work has focused on performing a comprehensive analysis of Tweets about this ongoing outbreak. To address these challenges, this work makes three scientific contributions to this field. First, it presents an open-access dataset of 556,427 Tweets about monkeypox that have been posted on Twitter since the first detected case of this outbreak. A comparative study is also presented that compares this dataset with 36 prior works in this field that focused on the development of Twitter datasets to further uphold the novelty, relevance, and usefulness of this dataset. Second, the paper reports the results of a comprehensive analysis of the Tweets of this dataset. This analysis presents several novel findings; for instance, out of all the 34 languages supported by Twitter, English has been the most used language to post Tweets about monkeypox, about 40,000 Tweets related to monkeypox were posted on the day WHO declared monkeypox as a GPHE, a total of 5470 distinct hashtags have been used on Twitter about this outbreak out of which #monkeypox is the most used hashtag, and Twitter for iPhone has been the leading source of Tweets about the outbreak. The sentiment analysis of the Tweets was also performed, and the results show that despite a lot of discussions, debate, opinions, information, and misinformation, on Twitter on various topics in this regard, such as monkeypox and the LGBTQI+ community, monkeypox and COVID-19, vaccines for monkeypox, etc., "neutral" sentiment was present in most of the Tweets. It was followed by "negative" and "positive" sentiments, respectively. Finally, to support research and development in this field, the paper presents a list of 50 open research questions related to the outbreak in the areas of Big Data, Data Mining, Natural Language Processing, and Machine Learning that may be investigated based on this dataset. [ABSTRACT FROM AUTHOR]

Authorship Analysis (AA) is a natural language processing field that examines the previous works of writers to identify the author of a text based on its features. Studies in authorship analysis include authorship identification, authorship profiling, and authorship verification. Due to its relevance, to many applications in this field attention has been paid. It is widely used in the attribution of historical literature. Other applications include legal linguistics, criminal law, forensic investigations, and computer forensics. This paper aims to provide an overview of the work done and the techniques applied in the authorship analysis domain. The examination of recent developments in this field is the principal focus. Many different criteria can be used to define a writer's style. This paper investigates stylometric features in different author-related tasks, including lexical, syntactic, semantic, structural, and content-specific ones. A lot of classification methods have been applied to authorship analysis tasks. We examine many research studies that use different machine learning and deep learning techniques. As a means of pointing the direction for future studies, we present the most relevant methods recently proposed. The reviewed studies include documents of different types and different languages. In summary, due to the fact that each natural language has its own set of features, there is no standard technique generically applicable for solving the AA problem. [ABSTRACT FROM AUTHOR]

"Fake news" refers to the deliberate dissemination of news with the purpose to deceive and mislead the public. This paper assesses the accuracy of several Machine Learning (ML) algorithms, using a style-based technique that relies on textual information extracted from news, such as part of speech counts. To expand the already proposed styled-based techniques, a new method of enhancing a linguistic feature set is proposed. It combines Named Entity Recognition (NER) with the Frequent Pattern (FP) Growth association rule mining algorithm, aiming to provide better insight into the papers' sentence level structure. Recursive feature elimination was used to identify a subset of the highest performing linguistic characteristics, which turned out to align with the literature. Using pre-trained word embeddings, document embeddings and weighted document embeddings were constructed using each word's TF-IDF value as the weight factor. The document embeddings were mixed with the linguistic features providing a variety of training/test feature sets. For each model, the best performing feature set was identified and fine-tuned regarding its hyper parameters to improve accuracy. ML algorithms' results were compared with two Neural Networks: Convolutional Neural Network (CNN) and Long-Short-Term Memory (LSTM). The results indicate that CNN outperformed all other methods in terms of accuracy, when companied with pre-trained word embeddings, yet SVM performs almost the same with a wider variety of input feature sets. Although style-based technique scores lower accuracy, it provides explainable results about the author's writing style decisions. Our work points out how new technologies and combinations of existing techniques can enhance the style-based approach capturing more information. [ABSTRACT FROM AUTHOR]

The knowledge graph was first used in the information search of the Internet as a way to improve the quality of the search because it contains a huge amount of structured knowledge data. In this paper, the knowledge map algorithm is studied through natural language processing technology and probabilistic fuzzy information aggregation, and the knowledge map completion algorithm is cognitive-fitted. NLP is natural language processing. Based on the experiments in this paper, it can be seen that, after combining the algorithm, the behavior data set of 1000 Amazon users was analyzed, and it can be found that the accuracy of the algorithm improves as the proportion of data in the experiment increases. Among them, the 10% dataset has a correct rate of 0.66; the 30% dataset has a final accuracy rate of 0.68; and the 50% dataset has a final accuracy rate of 0.70. The experimental results of this paper show that using probabilistic fuzzy information aggregation and natural language processing technology as a way to complete the knowledge graph can improve the accuracy of the operation. It plays an important role in the development of intelligent cognition and search engines. [ABSTRACT FROM AUTHOR]

System logs are an important data source for performance monitoring and anomaly detection. Analyzing logs for anomaly detection can improve service quality. At present, although machine learning algorithms for anomaly detection can achieve high accuracy, they lack robustness. The detection model cannot dynamically adapt to changes of logs when system logs contain noises owing to the casualness of the operators or log templates update. In face of this challenge, the paper proposes a robust log anomaly detection method based on contrastive learning and multi-scale Masked Sequence to Sequence (MASS). First, a log feature extraction model integrating the BERT model with contrastive learning is proposed. It can extract effective features by pulling two related normal logs together and pushing apart normal and abnormal logs to ensure that the semantic similarity between normal and abnormal log templates is lower than that between normal log templates, effectively remove abnormal log templates and distinguish log categories to which normal log templates and normal noise log templates belong rather than rudely treating the noise log templates as anomalies, which enhances the robustness of anomaly detection. Then, a multi-scale Masked Sequence to Sequence (MSMASS) model is proposed, the Attention mechanism of the MASS model is replaced with multi-scale Attention to fully learn the context information of different scales of the log sequence, which improves the accuracy of anomaly detection. Contrast experiments are conducted with four baseline methods on common datasets, and the results show that the method proposed in this paper is superior to most existing log-based anomaly detection methods in terms of accuracy and robustness. [ABSTRACT FROM AUTHOR]

Social work case files hold rich detail about the lives and needs of vulnerable groups. Traditional case-reading studies to gain generalisable knowledge are resource-intensive, however, and sample sizes thereby limited. The advent of 'big data' technology, and vast repositories of centrally stored electronic records offer social work researchers novel alternatives, including data linkage and predictive risk modelling using administrative data. Free-text documents, however – including assessments, reports, and case chronologies – remain a largely untapped resource. This paper describes how 5000 social work court statements held by the Child and Family Court Advisory Support Service in England (Cafcass) were analysed using natural language processing (NLP) based on simple rules and mathematical principles. Thirteen factors relating to harm and risk to children involved in care proceedings in England were identified by automated computer techniques, and almost 90% agreement with professional readers achieved when the factors were clear-cut. The study represents an innovative approach for social work research on complex social problems. In conclusion, the paper discusses learning points; practical implications; future research avenues; and the technical and ethical challenges of NLP. [ABSTRACT FROM AUTHOR]

In image captioning models, the main challenge in describing an image is identifying all the objects by precisely considering the relationships between the objects and producing various captions. Over the past few years, many methods have been proposed, from an attribute-to-attribute comparison approach to handling issues related to semantics and their relationships. Despite the improvements, the existing techniques suffer from inadequate positional and geometrical attributes concepts. The reason is that most of the abovementioned approaches depend on Convolutional Neural Networks (CNNs) for object detection. CNN is notorious for failing to detect equivariance and rotational invariance in objects. Moreover, the pooling layers in CNNs cause valuable information to be lost. Inspired by the recent successful approaches, this paper introduces a novel framework for extracting meaningful descriptions based on a parallelized capsule network that describes the content of images through a high level of understanding of the semantic contents of an image. The main contribution of this paper is proposing a new method that not only overrides the limitations of CNNs but also generates descriptions with a wide variety of words by using Wikipedia. In our framework, capsules focus on the generation of meaningful descriptions with more detailed spatial and geometrical attributes for a given set of images by considering the position of the entities as well as their relationships. Qualitative experiments on the benchmark dataset MS-COCO show that our framework outperforms state-of-the-art image captioning models when describing the semantic content of the images. [ABSTRACT FROM AUTHOR]

The article discusses various reports published within the issue, including one on the efficient grid scheduling, and another on planning for natural language generation (LNG).

Scientific research plays an important role in the achievement of a sustainable society. However, grasping the trends in sustainability research is difficult because studies are not devised and conducted in a top-down manner with Sustainable Development Goals (SDGs). To understand the bottom-up research activities, we analyzed over 300,000 publications concerned with sustainability by using citation network analysis and natural language processing. The results suggest that sustainability science's diverse and dynamic changes have been occurring over the last few years; several new topics, such as nanocellulose and global health, have begun to attract widespread scientific attention. We further examined the relationship between sustainability research subjects and SDGs and found significant correspondence between the two. Moreover, we extracted SDG topics that were discussed following a convergent approach in academic studies, such as "inclusive society" and "early childhood development", by observing the convergence of terms in the citation network. These results are valuable for government officials, private companies, and academic researchers, empowering them to understand current academic progress along with research attention devoted to SDGs. [ABSTRACT FROM AUTHOR]

The uniqueness of aesthetic implication in Zhou Dynasty poetics lies in that it is the basic forming stage of the concept of formal beauty of the whole Chinese nation, and the aesthetic implication of the Zhou Dynasty poetics art has fundamental significance for the whole ancient Chinese aesthetic implication theory. In the discipline of natural language processing, text emotion analysis is a crucial topic. Artificial neural network research is where the idea of "deep learning" (DL) first emerged. In view of the problems that semantic information is easy to be lost and emotional information may be ignored in the traditional Chinese short text emotion analysis model, this paper introduces the AM (attention mechanism) and proposes a CNN-LSTM (convolutional neural network-long short-term memory) poetic aesthetic implication analysis method based on self-attention. For the IL (input layer), word vectors trained by Word2Vec are used and then input into the CNN-LSTM joint model. Then, the output of the joint model is weighted and summed by self-attention and finally input into the Softmax classifier, so as to realize the emotion classification of the text. By creating and putting into practise pertinent comparative experiments, the usefulness of the proposed model is confirmed. The outcomes demonstrate that this model outperforms the other three comparison models for the quantification of evaluation indices in terms of overall performance. The accuracy and F1 of this paper are 93.362% and 90.886%, respectively, which are higher than other models. [ABSTRACT FROM AUTHOR]

The question of whether it is appropriate to attribute authorship to deceased individuals of original studies in the biomedical literature is contentious. Authorship guidelines utilized by journals do not provide a clear consensus framework that is binding on those in the field. To guide and inform the implementation of authorship frameworks it would be useful to understand the extent of the practice in the scientific literature, but studies that have systematically quantified the prevalence of this phenomenon in the biomedical literature have not been performed to date. To address this issue, we quantified the prevalence of publications by deceased authors in the biomedical literature from the period 1990–2020. We screened 2,601,457 peer-reviewed papers from the full text Europe PubMed Central database. We applied natural language processing, stringent filtering and manual curation to identify a final set of 1,439 deceased authors. We then determined these authors published a total of 38,907 papers over their careers with 5,477 published after death. The number of deceased publications has been growing rapidly, a 146-fold increase since the year 2000. This rate of increase was still significant when accounting for the growing total number of publications and pool of authors. We found that more than 50% of deceased author papers were first submitted after the death of the author and that over 60% of these papers failed to acknowledge the deceased authors status. Most deceased authors published less than 10 papers after death but a small pool of 30 authors published significantly more. A pool of 266 authors published more than 90% of their total publications after death. Our analysis indicates that the attribution of deceased authorship in the literature is not an occasional occurrence but a burgeoning trend. A consensus framework to address authorship by deceased scientists is warranted. [ABSTRACT FROM AUTHOR]

Higher education question and answer system is a current research hotspot in the field of natural language processing and artificial intelligence, which can return accurate answers directly and also allow users to input in natural language, avoiding the need to return large collections of valid and invalid data links for users to filter twice. In recent years, deep learning techniques have made great progress in the field of natural language processing, thus making it possible to apply it in the field of teaching and learning in junior high school. Most of the traditional quiz systems based on traditional retrieval techniques have problems such as insufficient semantic portrayal of text, inability to extract semantic features in context, and poor processing of complex utterances. In order to fill the gap and deficiencies of question and answer systems in junior high school teaching, this paper applies multimodal and adversarial network-related technologies to build a multimodal adversarial network-based question and answer system for higher education. The experimental results show that the principles of the traditional application system and the multimodal adversarial system designed in this paper are similar, but the current application system is relatively more effective. [ABSTRACT FROM AUTHOR]

Identifying possible research gaps is a main step in problem framing, however it is increasingly tedious and expensive considering the continuously growing amount of published material. This situation suggests the critical need for methodologies and tools that can assist researchers in their selection of future research topics. Related work mostly focuses on trend analysis and impact prediction but less on research gap identification. This paper presents our first approach in automated identification of feasible research gaps by using a double-threshold procedure to eliminate the research gaps that are currently difficult to study or offer little novelty. Gaps are then found by extracting subgraphs for the less-frequent co-occurrences and correlations of key terms describing domains. A case study applying the methodology for electronic design automation (EDA) domain is also discussed in the paper. [ABSTRACT FROM AUTHOR]

Nowadays, individuals have very stressful lifestyles, affecting their nutritional habits. In the early stages of life, teenagers begin to exhibit bad habits and inadequate nutrition. Likewise, other people with dementia, Alzheimer's disease, or other conditions may not take food or medicine regularly. Therefore, the ability to monitor could be beneficial for them and for the doctors that can analyze the patterns of eating habits and their correlation with overall health. Many sensors help accurately detect food intake episodes, including electrogastrography, cameras, microphones, and inertial sensors. Accurate detection may provide better control to enable healthy nutrition habits. This paper presents a systematic review of the use of technology for food intake detection, focusing on the different sensors and methodologies used. The search was performed with a Natural Language Processing (NLP) framework that helps screen irrelevant studies while following the PRISMA methodology. It automatically searched and filtered the research studies in different databases, including PubMed, Springer, ACM, IEEE Xplore, MDPI, and Elsevier. Then, the manual analysis selected 30 papers based on the results of the framework for further analysis, which support the interest in using sensors for food intake detection and nutrition assessment. The mainly used sensors are cameras, inertial, and acoustic sensors that handle the recognition of food intake episodes with artificial intelligence techniques. This research identifies the most used sensors and data processing methodologies to detect food intake. [ABSTRACT FROM AUTHOR]

Sarcasm is an integral part of human language and culture. Naturally, it has garnered great interest from researchers from varied fields of study, including Artificial Intelligence, especially Natural Language Processing. Automatic sarcasm detection has become an increasingly popular topic in the past decade. The research conducted in this paper presents, through a systematic literature review, the evolution of the automatic sarcasm detection task from its inception in 2010 to the present day. No such work has been conducted thus far and it is essential to establish the progress that researchers have made when tackling this task and, moving forward, what the trends are. This study finds that multi-modal approaches and transformer-based architectures have become increasingly popular in recent years. Additionally, this paper presents a critique of the work carried out so far and proposes future directions of research in the field. [ABSTRACT FROM AUTHOR]

Emotion recognition is an aspect of human-computer interaction, affective computing, and social robotics. Conventional unimodal approaches for emotion recognition, depending on single data sources such as facial expressions or speech signals often fall short in capturing the complexity and context-dependent nature of emotions. Multimodal Emotion Recognition (MER), which integrates information from multiple modalities, has emerged as a promising solution to overcome these limitations. In recent years, Transformers-based approaches have gathered significant attention in the fields of natural language processing and computer vision, highlighting their ability to capture long-range dependencies and semantic representations. These models have rapidly achieved the MER state-of-the-art. However, current survey papers that cover MER lack a specific focus on Transformer-based techniques. To bridge this research gap, this review paper provides a comprehensive investigation of Transformers-based approaches for MER. It explores various Transformer architectures and proposes several scenarios for using Transformers at different stages of MER process. In addition, it examines datasets suitable for MER, discusses fusion mechanisms, and introduces novel taxonomies in both MER and Transformer technologies. The review also addresses challenges and future research directions. Through this review, we aim to provide researchers with an inclusive understanding of the current state-of-the-art in Transformers-based approaches for MER, paving the way for further advancements in this rapidly developing field. • First specialized survey in transformer-based Multimodal Emotion Recognition. • Organized taxonomy of fusion techniques based on discerning criteria. • Transformers taxonomy categorized by structural and operational distinctions. • Diverse scenarios of applying transformers at different stages of MER process. • Meticulous analysis identifies trends and challenges across multimodal datasets. [ABSTRACT FROM AUTHOR]

The advancement of deep learning and neural networks has led to the widespread adoption of neural word embeddings as a prominent lexical representation method in natural language processing. With the help of the neural language model trained by the contextual information of large scale text, the neural word embedding obtained by the neural language model captures more semantic correlation in the semantic space, while ignoring the semantic similarity. It will incur high computational cost and time costs during the training process of the model. To better inject semantic similarity into the distribution space and reduce time cost, we perform post processing learning of neural word embeddings using deep metric learning. This paper proposes a lexical enhancement method based on flexible margins and multiple samples learning. In this method, we embed the lexical entailment constraint relations into neural word embeddings. By categorizing the set of lexical constraints and penalizing the negative samples to different degrees according to the gap between categories, and allowing the positive and negative samples to learn from each other in the distributed space. The method we propose significantly improves neural word embeddings. By evaluating neural word embedded vocabulary similarity, the benchmark accuracy is improved to 75%. The method shows great competitiveness in text similarity tasks and text categorization tasks. These findings summarize research results and provide strong support for further applications. • This paper proposes the LexFMSL model to optimize word embeddings. • This paper enhances LEAR model performance through flexible margins and learning methods. • The LexFMSL model achieves good results on different word embedding datasets. [ABSTRACT FROM AUTHOR]

In recent years, transformer-based models have emerged as powerful tools for natural language processing tasks, demonstrating remarkable performance in several domains. However, they still present significant limitations. These shortcomings become more noticeable when dealing with highly specific and complex concepts, particularly within the scientific domain. For example, transformer models have particular difficulties when processing scientific articles due to the domain-specific terminologies and sophisticated ideas often encountered in scientific literature. To overcome these challenges and further enhance the effectiveness of transformers in specific fields, researchers have turned their attention to the concept of knowledge injection. Knowledge injection is the process of incorporating outside knowledge into transformer models to improve their performance on certain tasks. In this paper, we present a comprehensive study of knowledge injection strategies for transformers within the scientific domain. Specifically, we provide a detailed overview and comparative assessment of four primary methodologies, evaluating their efficacy in the task of classifying scientific articles. For this purpose, we constructed a new benchmark including both 24K labelled papers and a knowledge graph of 9.2K triples describing pertinent research topics. We also developed a full codebase to easily re-implement all knowledge injection strategies in different domains. A formal evaluation indicates that the majority of the proposed knowledge injection methodologies significantly outperform the baseline established by Bidirectional Encoder Representations from Transformers. [ABSTRACT FROM AUTHOR]

In the current landscape, user opinions exert an unprecedented influence on the trajectory of companies. In the field of online review platforms, these opinions, transmitted through text reviews and numerical ratings, significantly shape the credibility of products and services. For this reason, detecting inappropriate reviews becomes crucial. This paper addresses the problem of automatic anomalous review detection using a novel approach based on Anomaly Detection in the field of Natural Language Processing (NLP). Unlike other NLP tasks, anomaly detection in texts is a relatively emerging area. In this paper, we present a pipeline for opinion filtering that poses the problem of discerning between normal opinions containing relevant information about an item and anomalous opinions with unrelated content. Its key functionalities include: Classifying the reviews, assigning normality scores, and generating explanations for each classification, indispensable for the human who normally moderates these platforms. To evaluate the model, several Amazon datasets were used to demonstrate that the performance obtained is robust, obtaining an average F1 score of 91.4 detecting anomalies in the most complex scenario. In addition, a comparative study of three explainability techniques was conducted with 241 participants to measure the impact on understanding the classifications of the model and to rank their perceived usefulness of explanations. As a result, we obtained a system with great potential to automate tasks related to online review platforms, offering insights into anomaly detection applications in textual data and showing the difficulties that arise when the task to be explained presents a subjectivity component. • A pipeline to detect anomalous reviews on online platforms. • Reviews are coded using a pretrained MPNet transformer model. • Anomaly detection is carried out using non-iterative autoencoder networks. • Classifications are accompanied by a normality score and an ad-hoc explanation. • A study has been carried out with 241 people to evaluate the explanations. [ABSTRACT FROM AUTHOR]

Emotion recognition is an aspect of human-computer interaction, affective computing, and social robotics. Conventional unimodal approaches for emotion recognition, depending on single data sources such as facial expressions or speech signals often fall short in capturing the complexity and context-dependent nature of emotions. Multimodal Emotion Recognition (MER), which integrates information from multiple modalities, has emerged as a promising solution to overcome these limitations. In recent years, Transformers-based approaches have gathered significant attention in the fields of natural language processing and computer vision, highlighting their ability to capture long-range dependencies and semantic representations. These models have rapidly achieved the MER state-of-the-art. However, current survey papers that cover MER lack a specific focus on Transformer-based techniques. To bridge this research gap, this review paper provides a comprehensive investigation of Transformers-based approaches for MER. It explores various Transformer architectures and proposes several scenarios for using Transformers at different stages of MER process. In addition, it examines datasets suitable for MER, discusses fusion mechanisms, and introduces novel taxonomies in both MER and Transformer technologies. The review also addresses challenges and future research directions. Through this review, we aim to provide researchers with an inclusive understanding of the current state-of-the-art in Transformers-based approaches for MER, paving the way for further advancements in this rapidly developing field. • First specialized survey in transformer-based Multimodal Emotion Recognition. • Organized taxonomy of fusion techniques based on discerning criteria. • Transformers taxonomy categorized by structural and operational distinctions. • Diverse scenarios of applying transformers at different stages of MER process. • Meticulous analysis identifies trends and challenges across multimodal datasets. [ABSTRACT FROM AUTHOR]

In recent years, transformer-based models have emerged as powerful tools for natural language processing tasks, demonstrating remarkable performance in several domains. However, they still present significant limitations. These shortcomings become more noticeable when dealing with highly specific and complex concepts, particularly within the scientific domain. For example, transformer models have particular difficulties when processing scientific articles due to the domain-specific terminologies and sophisticated ideas often encountered in scientific literature. To overcome these challenges and further enhance the effectiveness of transformers in specific fields, researchers have turned their attention to the concept of knowledge injection. Knowledge injection is the process of incorporating outside knowledge into transformer models to improve their performance on certain tasks. In this paper, we present a comprehensive study of knowledge injection strategies for transformers within the scientific domain. Specifically, we provide a detailed overview and comparative assessment of four primary methodologies, evaluating their efficacy in the task of classifying scientific articles. For this purpose, we constructed a new benchmark including both 24K labelled papers and a knowledge graph of 9.2K triples describing pertinent research topics. We also developed a full codebase to easily re-implement all knowledge injection strategies in different domains. A formal evaluation indicates that the majority of the proposed knowledge injection methodologies significantly outperform the baseline established by Bidirectional Encoder Representations from Transformers. [ABSTRACT FROM AUTHOR]

In the current landscape, user opinions exert an unprecedented influence on the trajectory of companies. In the field of online review platforms, these opinions, transmitted through text reviews and numerical ratings, significantly shape the credibility of products and services. For this reason, detecting inappropriate reviews becomes crucial. This paper addresses the problem of automatic anomalous review detection using a novel approach based on Anomaly Detection in the field of Natural Language Processing (NLP). Unlike other NLP tasks, anomaly detection in texts is a relatively emerging area. In this paper, we present a pipeline for opinion filtering that poses the problem of discerning between normal opinions containing relevant information about an item and anomalous opinions with unrelated content. Its key functionalities include: Classifying the reviews, assigning normality scores, and generating explanations for each classification, indispensable for the human who normally moderates these platforms. To evaluate the model, several Amazon datasets were used to demonstrate that the performance obtained is robust, obtaining an average F1 score of 91.4 detecting anomalies in the most complex scenario. In addition, a comparative study of three explainability techniques was conducted with 241 participants to measure the impact on understanding the classifications of the model and to rank their perceived usefulness of explanations. As a result, we obtained a system with great potential to automate tasks related to online review platforms, offering insights into anomaly detection applications in textual data and showing the difficulties that arise when the task to be explained presents a subjectivity component. • A pipeline to detect anomalous reviews on online platforms. • Reviews are coded using a pretrained MPNet transformer model. • Anomaly detection is carried out using non-iterative autoencoder networks. • Classifications are accompanied by a normality score and an ad-hoc explanation. • A study has been carried out with 241 people to evaluate the explanations. [ABSTRACT FROM AUTHOR]

Over the last two decades, many efforts have been made to provide resources to support the Arabic Natural Language Processing (NLP). Some of these resources target specific NLP tasks such as word tokenization, parsing, or sentiment analysis, while others attempt to tackle numerous tasks at once. In this paper, we present ¡¡TTK¿¿, a toolkit for Tunisian linguistic analysis. It consists of a collection of linguistic analysis tools for orthographic normalization, sentence boundaries detection, word tokenization, morphological analysis, parsing and named entity recognition. This paper focuses on the design and implementation of TTK tools. • NLP tools give us a better understanding of how the language may work in specific situations. • TTK contains text processing tools from orthographic normalization to comprehension. • All the proposed and used tools showed encouraging results. • All the proposed and used tools trait all the three forms of Tunisian dialect. [ABSTRACT FROM AUTHOR]

When writing academic papers, referencing statements, claims and previous studies is always an important activity. However, it is considered challenging for scientists to find relevant and appropriate scientific articles which are closely related to their current works in order to reference them in their research. As a consequence of the rapid growth in scientific papers being published every year, researchers might easily get overwhelmed within a huge number of resources. One way to help them find the desired references more easily is to use context-aware citation recommendation. The citation recommender system can automatically provide a list of suitable papers as references based on specified inputs which reflect the researchers' interests. Among the outstanding achievements of deep learning and natural language processing in recent years, the utilization of deep neural learning architectures have supported to address the problem of citation recommendation. As the result, the neural citation recommendation area has received much attention from the academic community, with the aim of enhancing the precision and correctness of the results of existing citation recommendation systems. Following this research direction, in our paper we present a novel context-aware citation recommendation model, called RHN-DualLCR (Recurrent Highway Networks – Dual Local Citation Recommendation), which integrates Recurrent Highway Networks (RHN), an improved model of the original Bidirectional Long Short-Term Memory (BiLSTM) architecture, and uses a SciBERT-based (Science text of Bidirectional Encoder Representations from Transformers) embedding layer to build up the efficiency of the state-of-the-art local citation recommendation model, which enriches context representation with global information. Our research demonstrates its originality and relevance because we used one of the latest achievements of deep models (RHN model) and natural language processing (SciBERT) applied to the citation recommendation problem. We have conducted experiments on the RHN-DualLCR model on 3 widely known datasets for the citation recommendation problem: ACL-200 (Association for Computational Linguistics), ACL-600 and RefSeer, and also used 2 common evaluation standards Mean Reciprocal Rank (MRR) and the Recall@K (R@K for short) to evaluate the performance of our model. Experimental results show that our proposed model is 3% to 16% better than the original models or state-of-the-art models. [ABSTRACT FROM AUTHOR]

Transformers are Deep Neural Networks (DNN) that utilize a self-attention mechanism to capture contextual relationships within sequential data. Unlike traditional neural networks and variants of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM), Transformer models excel at managing long dependencies among input sequence elements and facilitate parallel processing. Consequently, Transformer-based models have garnered significant attention from researchers in the field of artificial intelligence. This is due to their tremendous potential and impressive accomplishments, which extend beyond Natural Language Processing (NLP) tasks to encompass various domains, including Computer Vision (CV), audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published, spotlighting the Transformer's contributions in specific fields, architectural disparities, or performance assessments, there remains a notable absence of a comprehensive survey paper that encompasses its major applications across diverse domains. Therefore, this paper addresses this gap by conducting an extensive survey of proposed Transformer models spanning from 2017 to 2022. Our survey encompasses the identification of the top five application domains for Transformer-based models, namely: NLP, CV, multi-modality, audio and speech processing, and signal processing. We analyze the influence of highly impactful Transformer-based models within these domains and subsequently categorize them according to their respective tasks, employing a novel taxonomy. Our primary objective is to illuminate the existing potential and future prospects of Transformers for researchers who are passionate about this area, thereby contributing to a more comprehensive understanding of this groundbreaking technology. • The paper presents a comprehensive survey on transformers for deep learning tasks. • The paper conducts a thorough analysis on highly effective models in five domains. • The paper classifies the models based on respective tasks using a proposed taxonomy. • The characteristics of the surveyed models are deeply explored and analyzed. • Future directions and challenges for transformer-based models are deciphered. [ABSTRACT FROM AUTHOR]

Background: Named Entity Recognition (NER) is a long-standing fundamental problem in various research fields of Natural Language Processing (NLP) and has been practiced in many application scenarios. However, the application results of NER methods in Chinese electronic medical records (EMRs) are not satisfactory, mainly due to the following two problems: (1) Existing methods do not take into account the impact of medical terminology on model recognition performance, resulting in poor model performance. (2) Existing methods do not fully utilize the Chinese language features contained in EMR, resulting in poor model robustness. Therefore, it is imminent to solve these two problems regarding the performance of the NER model for EMRs. Methods: In this paper, a TENER-based radical feature and entity augmentation model for NER in Chinese EMRs is proposed. The TENER model is first used in the pre-training stage to extract deep semantic information from each layer of the feature extractor. In the decoder part, the recognition of medical entity boundary and entity category are divided into two branch tasks. Results: We compare the overall performance of the proposed model with existing models on different datasets using the computed F1 score evaluation metric. The experimental results show that our model achieves the best F1 score of 82.67%, 74.37%, 70.16% on the CCKS2019, ERTCMM, and CEMR data sets. Meanwhile, in the CMeEE challenge, our model surpassed the top-3 with the F1 score of 68.39%. Conclusions: Our proposed model is the first to divide the NER task into a two-branch tasks, entity boundary and types recognition. Firstly, the medical entity dictionary information is integrated into TENER to obtain the feature information of professional terms in Chinese EMRs. Secondly, the features of Chinese radicals in Chinese EMRs extracted by CNN are added to the entity category recognition task. Finally, the effectiveness of the model is validated on four datasets and competitive results are achieved. [ABSTRACT FROM AUTHOR]

Machine learning (ML) has been universally adopted for automated decisions in a variety of fields, including recognition and classification applications, recommendation systems, natural language processing, and so on. However, in light of high expenses on training data and computing resources, recent years have witnessed a rapid increase in outsourced ML training, either partially or completely, which provides vulnerabilities for adversaries to exploit. A prime threat in training phase is called poisoning attack, where adversaries strive to subvert the behavior of machine learning systems by poisoning training data or other means of interference. Although a growing number of relevant studies have been proposed, the research among poisoning attack is still overly scattered, with each paper focusing on a particular task in a specific domain. In this survey, we summarize and categorize existing attack methods and corresponding defenses, as well as demonstrate compelling application scenarios, thus providing a unified framework to analyze poisoning attacks. Besides, we also discuss the main limitations of current works, along with the corresponding future directions to facilitate further researches. Our ultimate motivation is to provide a comprehensive and self-contained survey of this growing field of research and lay the foundation for a more standardized approach to reproducible studies. [ABSTRACT FROM AUTHOR]

In the increasingly complex domain of Korean voice phishing attacks, advanced detection techniques are paramount. Traditional methods have achieved some degree of success. However, they often fail to detect sophisticated voice phishing attacks, highlighting an urgent need for enhanced approaches to improve detection performance. Addressing this, we have designed and implemented a novel artificial neural network (ANN) architecture that successfully combines data-centric and model-centric AI methodologies for detecting Korean voice phishing attacks. This paper presents our unique hybrid architecture, consisting of a 1-dimensional Convolutional Neural Network (1D CNN), a Bidirectional Long Short-Term Memory (BiLSTM), and Hierarchical Attention Networks (HANs). Our evaluations using the real-world KorCCVi v2 dataset demonstrate that the proposed architecture effectively leverages the strengths of CNN and BiLSTM to extract and learn contextually rich features from word embedding vectors. Additionally, implementing word and sentence attention mechanisms from HANs enhances the model's focus on crucial features, considerably improving detection performance. Achieving an accuracy score of 99.32% and an F1 score of 99.31%, our model surpasses all baseline models we trained, outperforms several existing solutions, and maintains comparable performance to others. The findings of this study underscore the potential of hybrid neural network architectures in improving voice phishing detection in the Korean language and pave the way for future research. This could involve refining and expanding upon this model to tackle increasingly sophisticated voice phishing strategies effectively or utilizing larger datasets. [ABSTRACT FROM AUTHOR]

Problem solving applications require users to exercise caution in their data usage practices. Prior to installing these applications, users are encouraged to read and comprehend the terms of service, which address important aspects such as data privacy, processes, and policies (referred to as information elements). However, these terms are often lengthy and complex, making it challenging for users to fully grasp their content. Additionally, existing transparency analytics tools typically rely on the manual extraction of information elements, resulting in a time-consuming process. To address these challenges, this paper proposes a novel approach that combines information visualization and machine learning analyses to automate the retrieval of information elements. The methodology involves the creation and labeling of a dataset derived from multiple software terms of use. Machine learning models, including naïve Bayes, BART, and LSTM, are utilized for the classification of information elements and text summarization. Furthermore, the proposed approach is integrated into our existing visualization tool TranspVis to enable the automatic detection and display of software information elements. The system is thoroughly evaluated using a database-connected tool, incorporating various metrics and expert opinions. The results of our study demonstrate the promising potential of our approach, serving as an initial step in this field. Our solution not only addresses the challenge of extracting information elements from complex terms of service but also provides a foundation for future research in this area. [ABSTRACT FROM AUTHOR]

This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases by speech with intonation. The paper proposes an improved method of continuous recognition of sign language, the results of which are transmitted to a natural language processor based on analyzers of morphology, syntax, and semantics of the Kazakh language, including morphological inflection and the construction of an intonation model of simple sentences. This approach has significant practical and social significance, as it can lead to the development of technologies that will help people with disabilities to communicate and improve their quality of life. As a result of the cross-validation of the model, we obtained an average test accuracy of 0.97 and an average val_accuracy of 0.90 for model evaluation. We also identified 20 sentence structures of the Kazakh language with their intonational model. [ABSTRACT FROM AUTHOR]

Simple Summary: The rapidly advancing field of deep learning, specifically transformer-based architectures and attention mechanisms, has found substantial applicability in bioinformatics and genome data analysis. Given the analogous nature of genome sequences to language texts, these techniques initially successful in natural language processing have been applied to genomic data. This review provides an in-depth analysis of the most recent advancements and applications of these techniques to genome data, critically evaluating their advantages and limitations. By investigating studies from 2019 to 2023, this review identifies potential future research areas, thereby encouraging further advancements in the field. The emergence and rapid development of deep learning, specifically transformer-based architectures and attention mechanisms, have had transformative implications across several domains, including bioinformatics and genome data analysis. The analogous nature of genome sequences to language texts has enabled the application of techniques that have exhibited success in fields ranging from natural language processing to genomic data. This review provides a comprehensive analysis of the most recent advancements in the application of transformer architectures and attention mechanisms to genome and transcriptome data. The focus of this review is on the critical evaluation of these techniques, discussing their advantages and limitations in the context of genome data analysis. With the swift pace of development in deep learning methodologies, it becomes vital to continually assess and reflect on the current standing and future direction of the research. Therefore, this review aims to serve as a timely resource for both seasoned researchers and newcomers, offering a panoramic view of the recent advancements and elucidating the state-of-the-art applications in the field. Furthermore, this review paper serves to highlight potential areas of future investigation by critically evaluating studies from 2019 to 2023, thereby acting as a stepping-stone for further research endeavors. [ABSTRACT FROM AUTHOR]

Recent advancements in Natural Language Processing (NLP), particularly in Large Language Models (LLMs), associated with deep learning-based computer vision techniques, have shown substantial potential for automating a variety of tasks. These are known as Visual LLMs and one notable model is Visual ChatGPT, which combines ChatGPT's LLM capabilities with visual computation to enable effective image analysis. These models' abilities to process images based on textual inputs can revolutionize diverse fields, and while their application in the remote sensing domain remains unexplored, it is important to acknowledge that novel implementations are to be expected. Thus, this is the first paper to examine the potential of Visual ChatGPT, a cutting-edge LLM founded on the GPT architecture, to tackle the aspects of image processing related to the remote sensing domain. Among its current capabilities, Visual ChatGPT can generate textual descriptions of images, perform canny edge and straight line detection, and conduct image segmentation. These offer valuable insights into image content and facilitate the interpretation and extraction of information. By exploring the applicability of these techniques within publicly available datasets of satellite images, we demonstrate the current model's limitations in dealing with remote sensing images, highlighting its challenges and future prospects. Although still in early development, we believe that the combination of LLMs and visual models holds a significant potential to transform remote sensing image processing, creating accessible and practical application opportunities in the field. [ABSTRACT FROM AUTHOR]

As an emerging business model, live broadcast e-commerce is growing rapidly in China in recent years and prompts the emergence of network anchors, attracting the customers to buy a variety of goods in their live broadcasts. In this paper, we take Li Jiaqi, one of most popular live streaming anchors in China as example, collect and transcribe his speech data of live broadcasts to build live broadcast speech-text corpus, and then conduct corpus-based analysis on linguistic characteristics from the aspects of phonetic, usage of various language units, and rhetoric ways, by using the methods and tools from Phonetics, Natural Language Processing, and Corpus Linguistics. Based on the linguistic analysis, we propose the hypothesis that features in the live broadcasts can have some effects on purchase intention of customers, and testify it with questionnaire survey. [ABSTRACT FROM AUTHOR]

Sentence classification is an important task in natural language processing. The task makes use of deep networks to enclose a mass of features with different granularities in a sentence. However, the classification usually suffer from severe performance degradation when stacking a large number of networks. The main reason is that, in a deep architecture, the silent feature representations are easily weakened and mixed with noisy information, which is not effective in learning contextual features and constructing semantic dependencies in a sentence. In this paper, a deep penetration network (DPN) is designed to improve deep architectures' ability to preserve the favourable semantic features. The DPN enables salient features to penetrate through a deeper architecture and to construct long semantic dependencies between them. This approach is evaluated on seven public datasets. Our experiments show that the DPN exhibits a stable performance with deeper architectures. It improves the performance on three types of sentence classification tasks, outperforming the existing state-of-the-art models. • A DPN network is proposed for learning semantic features of a sentence. • The DPN enables salient features to penetrate through deeper architectures. • It also enables to construct semantic dependencies between abstract features. • A DPN-based deep architecture is designed to support sentence classification. • Our model achieves the SOTA performance on seven public datasets. [ABSTRACT FROM AUTHOR]

Human prescription drug labeling contains a summary of the essential scientific information needed for the safe and effective use of the drug and includes the Prescribing Information, FDA-approved patient labeling (Medication Guides, Patient Package Inserts and/or Instructions for Use), and/or carton and container labeling. Drug labeling contains critical information about drug products, such as pharmacokinetics and adverse events. Automatic information extraction from drug labels may facilitate finding the adverse reaction of the drugs or finding the interaction of one drug with another drug. Natural language processing (NLP) techniques, especially recently developed Bidirectional Encoder Representations from Transformers (BERT), have exhibited exceptional merits in text-based information extraction. A common paradigm in training BERT is to pretrain the model on large unlabeled generic language corpora, so that the model learns the distribution of the words in the language, and then fine-tune on a downstream task. In this paper, first, we show the uniqueness of language used in drug labels, which therefore cannot be optimally handled by other BERT models. Then, we present the developed PharmBERT, which is a BERT model specifically pretrained on the drug labels (publicly available at Hugging Face). We demonstrate that our model outperforms the vanilla BERT, ClinicalBERT and BioBERT in multiple NLP tasks in the drug label domain. Moreover, how the domain-specific pretraining has contributed to the superior performance of PharmBERT is demonstrated by analyzing different layers of PharmBERT, and more insight into how it understands different linguistic aspects of the data is gained. [ABSTRACT FROM AUTHOR]

In this study, we will try to answer the question if an AI language model can provide teachers with essay assessment solutions that are on a par with the solutions provided by experienced professors. We designed a study with the aim of comparing the essay assessment outputs of the AI language model and three of our colleagues working at the University of Montenegro. The main aim of this paper is to investigate if this AI language model can be a viable teachers' assistance tool that provides immediate and meaningful feedback to teachers and students. Our hypothesis is, with some caveats, that the AI language model is more than a viable and useful tool, capable of providing meaningful and immediate feedback, greatly reducing the assessment time, and thus helping the teachers become more efficient and consistent. We will compare the results of 78 essays assessed by three teachers with the results provided by ChatGPT and see where the two sets of results converge or diverge in terms of their individual and overall scores. [ABSTRACT FROM AUTHOR]

Nowadays, the internet and social media ease the process of reviewing products. Consumers expose their thoughts, opinions, and experiences about products and services on various forums, websites, and mobile apps. In effect, internet reviews become a decision-maker for many people before getting their desired goods. Actually, text sentiment analysis consists of extracting insights and sentiments from social texts and consumers' reviews. Hence, various organizations conduct this analysis in order to better understand the attitude as well as the feedback of their customers toward their products. Besides, many scientific researchers are also interested in the analysis of customers' reviews by labeling them into a set of sentiments using some text classification algorithms. The following paper provides a convolutional neural network (CNN) model to classify text reviews' sentiments as negative or positive. Also, we make a comparative analysis using our proposed CNN model and several models' representations of word embedding to get the most efficient model. The experiments are implemented on the Amazon reviews dataset, and the diverse model designs have achieved appropriate performances. Reached results discern the importance of including stop-word in sentiment analysis tasks, in fact, the stop words elimination can provoke an inaccurate prediction of sentiments. Practically, using stop words with the CNN model has improved the accuracy result by 2% opposing the CNN model that has ignored them. Furthermore, we procure that the employment of a random initialization approach provides better performance than supervised and embedding model vectors on large-scale datasets. Effectively, training the representation of word embedding allows the model to learn better features in less computation time. Moreover, our CNN model showed better performance than the baseline machine learning and deep learning methods and improved the accuracy of the CNN to 90% on the Amazon reviews dataset. [ABSTRACT FROM AUTHOR]

Background: In recent years, the integration of Artificial Intelligence (AI) into various fields has led to transformative changes in research and practice. One such domain that has seen substantial impact is the medical field, where AI-powered tools like ChatGPT have emerged as potential alternatives to traditional methods for generating medical literature. This study presents a comprehensive comparative bibliometric analysis of medical literature generated by ChatGPT and traditional methods, aiming to uncover the emerging trends and potential shifts in the landscape. Materials and Methods: This study collects and analyzes a substantial corpus of articles, reviews, and papers generated by both ChatGPT and traditional methodologies. Bibliometric indicators such as publication frequency, citation counts, collaboration patterns, and keyword usage are examined to discern differences in output and impact. We extract data from Web of Science citation database and selected 18087 publications from the year 2019 to 2023 for our study. The data and descriptive analysis were categorised, collected one at a time, and imported into the Bibliometric R-package programme to produce science maps and statistical graphs. They were exported to MS-Excel for bibliometric analysis and VOSviewer software was used to analyse Co-Occurrence networks. Results and Discussion: A total of 18087 publications on ChatGPT and traditional methodologies from the year 2019 to 2023, namely 12519 (69.29%) original articles, 2836 (15%) reviews, 233 (01.2%) letters, and others. The most productive institution was found to be the Indian Institute of Technology System IIT System (n=1718, 0.09%), followed by National Institute of Technology NIT System (n=1275, 0.07%). the most productive author was found to be the Kumar, Atul, All India Institute of Medical Sciences (AIIMS) New Delhi (n=421, 2.39%), followed by Kumar, Satish, Indian Institute of Management Nagpur (n=405, 2.24%). The most productive journal was found that the IEEE Acess (n=373, 2.063%, TC=6740, ACP=18.06) followed by Multimedia Tools and Applications (n=279, 1.543%, TC=1552, ACP=5.56). The most frequent of authors keywords and occurrences was found that the 'artificial intelligence' 1517 occurrences and 1898 total link strength followed by 'deep learning' 1156 occurrences and 1764 total link strength. Conclusions: This bibliometric analysis sheds light on the evolving landscape of medical literature production, comparing the outputs of ChatGPT and traditional methods. While ChatGPT shows promise in its ability to quickly generate content on cutting-edge topics, traditional methods maintain their dominance in terms of research depth and impact. The findings have implications for researchers, clinicians, and policymakers, suggesting potential ways to leverage both approaches for a more comprehensive and impactful medical research ecosystem. Further research is warranted to monitor the trajectory of this evolving paradigm shift in medical literature and its long-term implications. [ABSTRACT FROM AUTHOR]

Networks are ubiquitous in today’s world. Community structure is a well-known feature of many empirical networks, and a lot of statistical methods have been developed for community detection. In this paper, we consider the problem of community extraction in text networks, which is greatly relevant in medical errors and patient safety databases. We adapt a well-known community extraction method to develop a scalable algorithm for extracting groups of similar documents in large text databases. The application of our method on a real-world patient safety report system demonstrates that the groups generated from community extraction are much more accurate than manual tagging by frontline workers. [ABSTRACT FROM AUTHOR]

Many scientific texts are produced in Persian and available in scientific information databases through the Web. In this paper, FarsAcademic, a test collection of Persian scientific texts, has been built to implement information retrieval models among academic search comprising 102238 documents and 61 topics. While constructing FarsAcademic, we have tried to resolve the problems specific to information retrieval (IR) and natural language processing (NLP) in Persian scientific texts. Domain experts were employed to create queries within their research area, and user relevance and topical relevance were applied to improve the precision of relevance judgment of documents. Further, to improve retrieval performance in Persian scientific texts, automated query expansion was applied using one of the relevant feedback techniques, the Local Context Analysis algorithm. The result showed that query expansion techniques outperformed other information retrieval models in the Persian scientific texts retrieval task. Eventually, FarsAcademic became the only one that has been free of charge for Iranian information retrieval scholars to implement and evaluate different information retrieval models and algorithms on Persian scientific text and academic search. [ABSTRACT FROM AUTHOR]

Abstract Meaning Representation parsing aims to represent a sentence as a structured, Directed, Acyclic Graph (DAG), in an attempt to extract meaning from text. This paper extends an existing 2-stage pipeline AMR parser with state-of-the-art techniques in dependency parsing. First, Pointer-Generator Networks are used for out-of-vocabulary words in the concept identification stage, with an improved initialization via the use of word-and character-level embeddings. Second, the performance of the Relation Identification module is improved by jointly training the Heads Selection and the Arcs Labeling components. Last, we underline the difficulty of end-to-end training with recurrent modules in a static deep neural network construction approach and explore a dynamic construction implementation, which continuously adapts the computation graph, thus potentially enabling end-to-end training in the proposed pipeline solution. [ABSTRACT FROM AUTHOR]

Prompt learning has improved the performance of language models by reducing the gap in language model training methods of pre-training and downstream tasks. However, extending prompt learning in language models pre-trained with unimodal data to multimodal sources is difficult as it requires additional deep-learning layers that cannot be attached. In the natural-language emotion-recognition task, improved emotional classification can be expected when using audio and text to train a model rather than only natural-language text. Audio information, such as voice pitch, tone, and intonation, can give more information that is unavailable in text to predict emotions more effectively. Thus, using both audio and text can enable better emotion prediction in speech emotion-recognition models compared to semantic information alone. In this paper, in contrast to existing studies that use multimodal data with an additional layer, we propose a method for improving the performance of speech emotion recognition using multimodal prompt learning with text-based pre-trained models. The proposed method is using text and audio information in prompt learning by employing a language model pre-trained on natural-language text. In addition, we propose a method to improve the emotion-recognition performance of the current utterance using the emotion and contextual information of the previous utterances for prompt learning in speech emotion-recognition tasks. The performance of the proposed method was evaluated using the English multimodal dataset MELD and the Korean multimodal dataset KEMDy20. Experiments using both the proposed methods obtained an accuracy of 87.49%, F 1 score of 44.16, and weighted F 1 score of 86.28. [ABSTRACT FROM AUTHOR]

The increasing amount of electronic legal documents represents a great opportunity for the development of intelligent computational systems for legal texts processing and classification. Most of these systems use classical machine learning and large datasets in English. This paper proposes an approach to automatically predict legal case outcome from written description of the events in Arabic using deep learning. An in‐house corpus from the decisions of the Moroccan Court of Cassation is built and used to train a deep learning model that predicts judgement. As the created corpus is of limited size, a new data augmentation method is proposed to boost the prediction performance. Two settings for text representation are tested, namely FastText and GloVe embeddings, and multiple deep learning models architectures are tested. The proposed approach succeeds in predicting judicial decisions of the Moroccan Court of Cassation with an accuracy of 80.51% on six classes. Even with a small dataset, the proposed data augmentation method was helpful in improving the overall models' performance. Despite the advancement in the area of legal judgement prediction over the years, this work is the first attempt to predict legal outcome using the documents of the Moroccan Cassation court. The corpus created in the context of this work will be made publicly available to the community. [ABSTRACT FROM AUTHOR]

Background: Research on health and sustainable development is growing at a pace such that conventional literature review methods appear increasingly unable to synthesize all relevant evidence. This paper employs a novel combination of natural language processing (NLP) and network science techniques to address this problem and to answer two questions: (1) how is health thematically interconnected with the Sustainable Development Goals (SDGs) in global science? (2) What specific themes have emerged in research at the intersection between SDG 3 ("Good health and well-being") and other sustainability goals? Methods: After a descriptive analysis of the integration between SDGs in twenty years of global science (2001–2020) as indexed by dimensions.ai, we analyze abstracts of articles that are simultaneously relevant to SDG 3 and at least one other SDG (N = 27,928). We use the top2vec algorithm to discover topics in this corpus and measure semantic closeness between these topics. We then use network science methods to describe the network of substantive relationships between the topics and identify 'zipper themes', actionable domains of research and policy to co-advance health and other sustainability goals simultaneously. Results: We observe a clear increase in scientific research integrating SDG 3 and other SDGs since 2001, both in absolute and relative terms, especially on topics relevant to interconnections between health and SDGs 2 ("Zero hunger"), 4 ("Quality education"), and 11 ("Sustainable cities and communities"). We distill a network of 197 topics from literature on health and sustainable development, with 19 distinct network communities – areas of growing integration with potential to further bridge health and sustainability science and policy. Literature focused explicitly on the SDGs is highly central in this network, while topical overlaps between SDG 3 and the environmental SDGs (12–15) are under-developed. Conclusion: Our analysis demonstrates the feasibility and promise of NLP and network science for synthesizing large amounts of health-related scientific literature and for suggesting novel research and policy domains to co-advance multiple SDGs. Many of the 'zipper themes' identified by our method resonate with the One Health perspective that human, animal, and plant health are closely interdependent. This and similar perspectives will help meet the challenge of 'rewiring' sustainability research to co-advance goals in health and sustainability. [ABSTRACT FROM AUTHOR]

Software engineering artifact extraction from natural language requirements without human intervention is a challenging task. Out of these artifacts, the use case plays a prominent role in software design and development. In the literature, most of the approaches are either semi-automated or necessitate formalism or make use of restricted natural language for the extraction of use cases from textual requirements. In this paper, we resolve the challenge of automated artifact extraction from natural language requirements. We propose an automated approach to generate use cases, actors, and their relationships from natural language requirements. Our proposed approach involves no human intervention or formalism. To automate the proposed approach, we have used Natural Language Processing and Network Science. Our proposed approach provides promising results for the extraction of use case elements from natural language requirements. We validate the proposed approach using several literature-based case studies. The proposed approach significantly improves the results in comparison to an existing approach. On average, the proposed approach achieves around 71.5% accuracy (F-Measure), whereas the baseline method achieves around 16% accuracy (F-Measure) on average. The evaluation of the proposed approach on the literature-based case studies shows its significance for the extraction of use case elements from natural language requirements. The approach reduces human effort in software design and development. [ABSTRACT FROM AUTHOR]

The prolonged COVID-19 pandemic has tied up significant medical resources, and its management poses a challenge for the public health care decision making. Accurate predictions of the hospitalizations are crucial for the decision makers to make informed decision for the medical resource allocation. This paper proposes a method named County Augmented Transformer (CAT). To generate accurate predictions of four-week-ahead COVID-19 related hospitalizations for every states in the United States. Inspired by the modern deep learning techniques, our method is based on a self-attention model (known as the transformer model) that is actively used in Natural Language Processing. Our transformer based model can capture both short-term and long-term dependencies within the time series while enjoying computational efficiency. Our model is a data based approach that utilizes the publicly available information including the COVID-19 related number of confirmed cases, deaths, hospitalizations data, and the household median income data. Our numerical experiments demonstrate the strength and the usability of our model as a potential tool for assisting the medical resources allocation. [ABSTRACT FROM AUTHOR]

Relation classification is a significant task within the field of natural language processing. Its objective is to extract and identify relations between two entities in a given text. Within the scope of this paper, we construct an artificial dataset (CS13K) for relation classification in the realm of cybersecurity and propose two models for processing such tasks. For any sentence containing two target entities, we first locate the entities and fine-tune the pre-trained BERT model. Next, we utilize graph attention networks to iteratively update word nodes and relation nodes. A new relation classification model is constructed by concatenating the updated vectors of word nodes and relation nodes. Our proposed model achieved exceptional performance on the SemEval-2010 task 8 dataset, surpassing previous approaches with a remarkable F1 value of 92.3%. Additionally, we propose the integration of a ranking-based voting mechanism into the existing model. Our best results are an F1 value of 92.5% on the SemEval-2010 task 8 dataset and a value 94.6% on the CS13K dataset. These findings highlight the effectiveness of our proposed models in tackling relation classification tasks. [ABSTRACT FROM AUTHOR]

ChatGPT is an AI-powered chatbot platform that enables human users to converse with machines. It utilizes natural language processing and machine learning algorithms, transforming how people interact with AI technology. ChatGPT offers significant advantages over previous similar tools, and its potential for application in various fields has generated attention and anticipation. However, some experts are wary of ChatGPT, citing ethical implications. Therefore, this paper shows that ChatGPT has significant potential to transform marketing and shape its future if certain ethical considerations are taken into account. First, we argue that ChatGPT-based tools can help marketers create content faster and potentially with quality similar to human content creators. It can also assist marketers in conducting more efficient research and understanding customers better, automating customer service, and improving efficiency. Then we discuss ethical implications and potential risks for marketers, consumers, and other stakeholders, that are essential for ChatGPT-based marketing; doing so can help revolutionize marketing while avoiding potential harm to stakeholders. [ABSTRACT FROM AUTHOR]

The tremendous increase in documents available on the Web has turned finding the relevant pieces of information into a challenging, tedious, and time-consuming activity. Text summarization is an important natural language processing (NLP) task used to reduce the reading requirements of text. Automatic text summarization is an NLP task that consists of creating a shorter version of a text document which is coherent and maintains the most relevant information of the original text. In recent years, automatic text summarization has received significant attention, as it can be applied to a wide range of applications such as the extraction of highlights from scientific papers or the generation of summaries of news articles. In this research project, we are focused mainly on abstractive text summarization that extracts the most important contents from a text in a rephrased form. The main purpose of this project is to summarize texts in German. Unfortunately, most pretrained models are only available for English. We therefore focused on the German BERT multilingual model and the BART monolingual model for English, with a consideration of translation possibilities. As the source of the experiment setup, took the German Wikipedia article dataset and compared how well the multilingual model performed for German text summarization when compared to using machine-translated text summaries from monolingual English language models. We used the ROUGE-1 metric to analyze the quality of the text summarization. [ABSTRACT FROM AUTHOR]

In this paper, we present a rigorous mathematical examination of generative pre-trained transformer (GPT) models and their autoregressive self-supervised learning mechanisms. We begin by defining natural language space and knowledge space, which are two key concepts for understanding the dimensionality reduction process in GPT-based large language models (LLMs). By exploring projection functions and their inverses, we establish a framework for analyzing the language generation capabilities of these models. We then investigate the GPT representation space, examining its implications for the models' approximation properties. Finally, we discuss the limitations and challenges of GPT models and their learning mechanisms, considering trade-offs between complexity and generalization, as well as the implications of incomplete inverse projection functions. Our findings demonstrate that GPT models possess the capability to encode knowledge into low-dimensional vectors through their autoregressive self-supervised learning mechanism. This comprehensive analysis provides a solid mathematical foundation for future advancements in GPT-based LLMs, promising advancements in natural language processing tasks such as language translation, text summarization, and question answering due to improved understanding and optimization of model training and performance. [ABSTRACT FROM AUTHOR]