Search

Aspect level sentiment classification aims to classify the sentiment polarity of a given aspect term or aspect category in a sentence. For sentiment classification towards a given aspect term, since a sentence may contain more than one aspect term, there may exist some opinions which are not the modifiers of the given aspect term. It is necessary to capture relevant opinion for a certain aspect term. Previous works use the relative distance between an aspect term and all other words in a sentence, in order to capture the nearest opinion of the aspect term. This can be infeasible when the sentence has a complex syntactic structure. In this paper, we detect the dependency relation between an aspect term and its related sentiment words in the dependency parse tree. Then, we integrate this relationship into CNN and Bi-LSTM respectively. Experiments show that the related sentiment features for an aspect term is helpful for models to discriminate its sentiment polarity, and our proposed models achieve state-of-the-art results among neural network models.

The ability to accurately predict human encounters can inspire a variety of promising applications, ranging from epidemiology to data forwarding in opportunistic networks. This work aims at designing a low cost, highly accurate human encounter prediction model based on Wi-Fi datasets. By leveraging the temporal dependency of human mobility, we propose the distributed human encounter prediction (DHEP) model, which uses the Wi-Fi access history and inferred contact information of only the person of interest to estimate future encounters of that person. We implement the proposed DHEP model using a recurrent neural network and a feed-forward neural network. An embedding model that learns the low-dimensional representation of a person's location is proposed to reduce the number of training parameters. The experimental results on two large Wi-Fi datasets show the proposed RNN-based DHEP model outperforms existing models, and achieves 87% to 91% accuracy based on University at Buffalo (UB) traces. We also compare DH P with the centralized human encounter prediction (CHEP) model, which gathers the contact history of all people for predicting future encounters. Despite a slightly lower performance than CHEP, DHEP has a low overhead and can protect data privacy.

Collaborative Filtering (CF) filters the flow of data that can be recommended, by a Recommender System (RS), to a target user according to his taste and his preferences. The target user’s profile is built based on his similarity with other users. For this reason, CF technique is very sensitive to the similarity measure used to quantify the dependency strength between two users (or two items). In this paper we provide an in-depth review on similarity measures used for CF-based RS. For each measure, we outline its fundamental background and we test its performance through an experimental study. Experiments are carried out on three standard datasets (MovieLens100k, MovieLens1M and Jester) and reveal many important conclusions. In fact, results show that ITR and IPWR are the most suitable similarity measures for a user-based RS while AMI is the best choice for an item-based RS. Evaluation metrics show that under the user-based approach, ITR obtains an MAE equal to 0.786 and 0.731 on MovieLens100k and MovieLens1M, respectively. Whereas, IPWR reach an MAE equal to 3.256 on Jester. Also, AMI gets under the item-based approach an MAE equal to 0.745, 0.724 and 3.281 on MovieLens100k, MovieLens1M and Jester, respectively.

The practice of backporting aims to bring the benefits of a bug or vulnerability fix from a higher to a lower release of a software package. When such a package adheres to semantic versioning, backports can be recognised as new releases in a lower major train. This is particularly useful in case a substantial number of software packages continues to depend on that lower major train. In this article, we study the backporting practices in four popular package distributions, namely Cargo, npm, Packagist and RubyGems. We observe that many dependent packages could benefit from backports provided by their dependencies. In particular, we find that a majority of security vulnerabilities affect more than one major train but are only fixed in the highest one, letting thousands of dependent packages exposed to the vulnerability. Despite that, we find that backporting updates is quite infrequent, and mostly practised by long-lived and more active packages for a variety of reasons.

Part-of-Speech (POS) tagging is a fundamental sequence labeling problem in Natural Language Processing. Recent deep learning sequential models combine the forward and backward word informatio for POS tagging. The information of contextual words to the current word play a vital role in capturing the non-continuous relationship. We have proposed Monotonic chunk-wise attention with CNN-GRU-Softmax (MCCGS), a deep learning architecture that adheres to these essential information. This architecture consists of Input Encoder (IE), encodes word and character-level, Contextual Encoder (CE), assigns the weightage to adjacent word and Disambiguator (D), which resolves intra-label dependencies as core components. Moreover, different morphological features have been integrated into the core components of MCCGS architecture as MCCGS-IE, MCCGS-CE and MCCGS-D. The MCCGS architecture is validated on the 21 languages from Universal Dependency (UD) treebank. The state-of-the-art models, Type constraints, Retrofitting, Distant Supervision from Disparate Sources and Position-aware Self Attention, MCCGS and its variants such as MCCGS-IE, MCCGS-CE and MCCGS-D are obtained mean accuracy 83.65 % , 81.29 % , 84.10 % , 90.18 % , 90.40 % , 91.40 % , 90.90 % , 92.30 % , respectively. The proposed model architecture provides state-of-the-art accuracy on the low resource languages as Marathi ( 93.58 % ), Tamil ( 87.50 % ), Telugu ( 96.69 % ) and Sanskrit ( 97.28 % ) from UD treebank and Hindi ( 95.64 % ) and Urdu ( 87.47 % ) from Hindi-Urdu multi-representational treebank.

Multiview fuzzy systems aim to deal with fuzzy modeling in multiview scenarios effectively and to obtain the interpretable model through multiview learning. However, current studies of multiview fuzzy systems still face several challenges, one of which is how to achieve efficient collaboration between multiple views when there are few labeled data. To address this challenge, this article explores a novel transductive multiview fuzzy modeling method. The dependency on labeled data is reduced by integrating transductive learning into the fuzzy model to simultaneously learn both the model and the labels using a novel learning criterion. Matrix factorization is incorporated to further improve the performance of the fuzzy model. In addition, collaborative learning between multiple views is used to enhance the robustness of the model. The experimental results indicate that the proposed method is highly competitive with other multiview learning methods.

Web services often impose inter-parameter dependencies that restrict the way in which two or more input parameters can be combined to form valid calls to the service. Unfortunately, current specification languages for web services like the OpenAPI Specification (OAS) provide no support for the formal description of such dependencies, which makes it hardly possible to automatically discover and interact with services without human intervention. In this article, we present an approach for the specification and automated analysis of inter-parameter dependencies in web APIs. We first present a domain-specific language, called Inter-parameter Dependency Language (IDL), for the specification of dependencies among input parameters in web services. Then, we propose a mapping to translate an IDL document into a constraint satisfaction problem (CSP), enabling the automated analysis of IDL specifications using standard CSP-based reasoning operations. Specifically, we present a catalogue of nine analysis operations on IDL documents allowing to compute, for example, whether a given request satisfies all the dependencies of the service. Finally, we present a tool suite including an editor, a parser, an OAS extension, a constraint programming-aided library, and a test suite supporting IDL specifications and their analyses. Together, these contributions pave the way for a new range of specification-driven applications in areas such as code generation and testing. Junta de Andalucía US-1264651 Ministerio de Ciencia, Innovación y Universidades RTI2018-101204-B-C21 Ministerio de Educación y Formación Profesional FPU17/04077

Understanding human language is one of the key themes of artificial intelligence. For language representation, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy texts and getting rid of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanisms for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. The proposed SG-Net is applied to typical Transformer encoders. Extensive experiments on popular benchmark tasks, including machine reading comprehension, natural language inference, and neural machine translation show the effectiveness of the proposed SG-Net design., The early version accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Journal extension of arXiv:1908.05147 (AAAI 2020)

As one of the most complex social problems around the world, crime may bring the risk of dying or losing property to the public if not handled properly. Crime prediction which aims at predicting crime incidents before they happen is of great importance to fight against crime. Previous studies are concerned primarily with day-level crime prediction and have certain limitations on modeling complex spatial–temporal-categorical dependency contained in the criminal activities as well as utilizing external factors to facilitate the forecast. In this paper, we develop a novel Neural Attentive framework for Hour-level Crime prediction (NAHC) to cope with these challenges. Specifically, we first adopt the priori knowledge-based data enhancement strategy to alleviate the zero-inflated issue raised in hour-level settings. Then, multi-graph convolutional networks are applied to capture spatial dependency from different aspects. After that, we integrate gated recurrent units with a temporal attention mechanism to jointly address temporal dependency and capture time-sensitive external factors. A categorical attention mechanism is proposed for dealing with categorical dependency and finally a fully connected network is utilized to generate the final prediction results. Extensive experiments on two real-world crime datasets demonstrate the effectiveness of our framework over the state-of-the-art comparing methods.

Aspect-based sentiment analysis (ABSA) is crucial for exploring user feedbacks and preferences on produces or services. Although numerous classical deep learning-based methods have been proposed in previous literature, several useful cues (e.g., contextual, lexical, and syntactic) are still not fully considered and utilized. In this study, a new approach for ABSA is proposed through the guidance of contextual, lexical, and syntactic cues. First, a novel sub-network is introduced to represent a target in a sentence in ABSA by considering the whole context. Second, lexicon embedding is applied to incorporate additional lexical cues. Third, a new attention module, namely, dependency attention, is proposed to elaborate syntactic dependency cues between words in attention inference. Experimental results on four benchmark data sets demonstrate the effectiveness of our proposed approach to aspect-based sentiment analysis.

With the widespread application of smart devices, spatial crowdsourcing (SC) has been extensively integrated into daily life. Task assignment is a crucial issue in SC and has attracted much attention. Most prior studies on task assignment ignore the importance of dependency among tasks, resulting in some ineffective matching pairs and wasting workers’ time. To this end, we formulate a new problem in SC, abbreviated as multi-stage complex task assignment (MSCTA), which aims to assign workers to multi-stage complex tasks to maximize the total profit. Compared with existing studies, MSCTA can obtain more effective assignments by considering the dependency constraints among tasks. We prove that the MSCTA problem is NP-hard and propose a greedy algorithm and a game algorithm. Specifically, both algorithms iteratively utilize a filtering module to obtain a set of executable tasks (ET) for assignment. The greedy algorithm can quickly assign the most profitable workers to the subtasks in each round of ET, and obtain a provable approximate result. The game algorithm is proved to be convergent and can win a Nash equilibrium when processing the subtasks in each round of ET. Extensive experimental results demonstrate the efficiency of our algorithm.

Chemical-induced disease (CID) relation extraction from biomedical articles plays an important role in disease treatment and drug development. Existing methods are insufficient for capturing complete document level semantic information due to ignoring semantic information of entities in different sentences. In this work, we proposed an effective document-level relation extraction model to automatically extract intra-/inter-sentential CID relations from articles. Firstly, our model employed BERT to generate contextual semantic representations of the title, abstract and shortest dependency paths (SDPs). Secondly, to enhance the semantic representation of the whole document, cross attention with self-attention (named cross2self-attention) between abstract, title and SDPs was proposed to learn the mutual semantic information. Thirdly, to distinguish the importance of the target entity in different sentences, the Gaussian probability distribution was utilized to compute the weights of the co-occurrence sentence and its adjacent entity sentences. More complete semantic information of the target entity is collected from all entities occurring in the document via our presented document-level R-BERT (DocR-BERT). Finally, the related representations were concatenated and fed into the softmax function to extract CIDs. We evaluated the model on the CDR corpus provided by BioCreative V. The proposed model without external resources is superior in performance as compared with other state-of-the-art models (our model achieves 53.5%, 70%, and 63.7% of the F1-score on inter-/intra-sentential and overall CDR dataset). The experimental results indicate that cross2self-attention, the Gaussian probability distribution and DocR-BERT can effectively improve the CID extraction performance. Furthermore, the mutual semantic information learned by the cross self-attention from abstract towards title can significantly influence the extraction performance of document-level biomedical relation extraction tasks.

Non-intrusive load monitoring (NILM) is to estimate individual appliance’s power consumption from aggregated smart meter data, which is useful for optimized energy management and provisioning of customized services. While deep learning (DL) has achieved state-of-the-art NILM performance, it is still constrained by the dependency on large amounts of data and intensive computations on training. In this paper, we propose a pre-training approach to address the generalization of DL models for NILM. We develop a meta-learning based approach and an ensemble learning based approach, which pre-train a base model and then fine-tune it with few-short learning when applied to an unknown dataset. The models are validated with two real-world datasets and shown to achieve a superior transferability performance compared with traditional DL and transfer learning methods.

A faulty situation occurred in Service Oriented Architecture (SOA) based systems can affect its functionality. It is challenging to guarantee the reliability of service utilization in a distributed, concurrent and dynamic composition of web services. This paper aims to present a fault detection approach of SOA using dependency analysis. Data and control dependencies are identified and a fault diagnosis approach based on these dependencies is proposed in this paper. The presented approach extends discrete state space techniques to simulate the concurrent and distributed behavior through Colored Petri Nets (CPNs) for the SOA-based systems in case of the faulty situation. Concrete translation from SOA based system to a CPN model is presented. Model-based technique is used and some heuristics are presented for dependency analysis and fault detection purpose. The experimental performance analysis shows the feasibility of our proposed approach.

Coronavirus Disease-19 (COVID-19) has lead global epidemics with high morbidity and mortality. However, there are currently no proven effective drugs targeting COVID-19. Identifying drug-virus associations can not only provide insights into the understanding of drug-virus interaction mechanism, but also guide and facilitate the screening of compound candidates for antiviral drug discovery. Since conventional experiment methods are time-consuming, laborious and expensive, computational methods to identify potential drug candidates for viruses (e.g., COVID-19) provide an alternative strategy. In this work, we propose a novel framework of Heterogeneous Graph Attention Networks for Drug-Virus Association predictions, named HGATDVA. First, we fully incorporate multiple sources of biomedical data, e.g., drug chemical information, virus genome sequences and viral protein sequences, to construct abundant features for drugs and viruses. Second, we construct two drug-virus heterogeneous graphs. For each graph, we design a self-enhanced graph attention network (SGAT) to explicitly model the dependency between a node and its local neighbors and derive the graph-specific representations for nodes. Third, we further develop a neural network architecture with tri-aggregator to aggregate the graph-specific representations to generate the final node representations. Extensive experiments were conducted on two datasets, i.e., DrugVirus and MDAD, and the results demonstrated that our model outperformed 7 state-of-the-art methods. Case study on SARS-CoV-2 validated the effectiveness of our model in identifying potential drugs for viruses.

Quality prediction, as the basis of quality control, is dedicated to predicting quality indices of the manufacturing process. In recent years, data-driven deep learning methods have received a lot of attention due to their accuracy, robustness, and convenience for the prediction of quality indices. However, the existing studies mainly focus on the quality prediction of a single machine, while ignoring dependency relationships among multiple machines in multistage manufacturing process. To tackle the above issues, a novel path enhanced bidirectional graph attention network (PGAT) is proposed in this article. PGAT models the dependencies among machines into directed graphs and introduces graph attention network to encode the dependencies. Nonetheless, it is difficult for graph neural networks to encode long-distance dependencies. Hence, dependency path information is introduced into the features of machines. Moreover, in order to solve the label noise problem that often occurs in actual industrial dataset, a masked loss function is devised. With its help, batch training with noisy labels can be achieved, which improves the training efficiency. Furthermore, experiments are conducted on a public quality prediction dataset collected from an actual production line. PGAT achieves the state-of-the-art results on this dataset, which confirms the effectiveness of PGAT. Additionally, the experimental results demonstrate the critical role of modeling dependency relationships among machines.

Reliable traffic flow prediction is of great value in the field of transportation, which, for example, contributes to traffic control and public safety. The key of achieving better performance is to well capture the non-linear spatial-temporal dependency. The state-of-the-art works consider both aspects, but they ignore the effect of the global trend on local dynamics and fail to capture long-term dynamic dependencies. In this article, we propose a novel Global-Local Temporal Convolutional Network (GL-TCN) to break through these limitations. Specifically, a novel local temporal convolutional mechanism is proposed to capture the long-term local dynamics effectively. Meanwhile, the global and local flow patterns are integrated to handle the effect of the global flow trend on local dynamics. To the best of our knowledge, this is the first work to utilize the temporal convolutional network for traffic flow prediction. Experiments on two real-world datasets demonstrate the superior performance of our method over several state-of-the-art baselines.

The blooming Internet of Things (IoTs) devices have brought many new types of sensing applications and methods to the traditional cloud-enabled IoTs framework. Hence, the new network framework becomes bidirectional gradually, in which IoTs devices can also perform moderate computation tasks instead of being solely used as data harvesters. This new coming framework known as edge computing significantly improves the traditional cloud-enabled network latency and dependency by shifting part of computation back to “local”. However, new security risks emerge when the edge computing shifts data and models back to the IoTs devices. Acies, a differential privacy based privacy-preserving classification system for edge computing is proposed to secure the classification models offloaded to edge devices. Acies supports popular classifiers such as Nearest Neighborhood, Support Vector Machine and Sparse Representation Classifier with a variety of feature selection methods. According to our evaluation on different datasets, classification models with Acies can be private and maintain high utility. Acies achieves reliable privacy protection under reconstruction attacks with minimal impact on classification accuracy (2% ∽ 5%) only. Acies outperforms the naive input dataset perturbation methods by up to 30% higher classification accuracy when the privacy requirements of the applications is high (the privacy budget e is less than 2).

Relation extraction is a critical natural language processing task. Existing dependency-based models captured long-range syntactic relations, but they usually cannot fully exploit information from sentences. They often used hand-crafted rules to prune redundant edges from dependency trees, but suffer from the imbalance of including and removing contents. When incorporating sequence models, they usually ignored the semantic and syntactic interactions between words. In this paper, we propose to automatically learn relational dependency structures with a fine-grained gating strategy. We decompose the dependency tree into differently informative parts and apply different gating methods to each part. To further capture the word-level interactions, we propose to apply the co-attention mechanism to combine structure and sequence models. We apply a neural network to learn the affinity matrix and derive mutual attention weights between semantic and syntactic representations. We conduct experiments on two benchmark datasets and the results indicate the effectiveness of our method.

Multimodal human sentiment comprehension refers to recognizing human affection from multiple modalities. There exist two key issues for this problem. Firstly, it is difficult to explore time-dependent interactions between modalities and focus on the important time steps. Secondly, processing the long fused sequence of utterances is susceptible to the forgetting problem due to the long-term temporal dependency. In this paper, we introduce a hierarchical learning architecture to classify utterance-level sentiment. To address the first issue, we perform time-step level fusion to generate fused features for each time step, which explicitly models time-restricted interactions by incorporating information across modalities at the same time step. Furthermore, based on the assumption that acoustic features directly reflect emotional intensity, we pioneer emotion intensity attention to focus on the time steps where emotion changes or intense affections take place. To handle the second issue, we propose Residual Memory Network (RMN) to process the fused sequence. RMN utilizes some techniques such as directly passing the previous state into the next time step, which helps to retain the information from many time steps ago. We show that our method achieves state-of-the-art performance on multiple datasets. Results also suggest that RMN yields competitive performance on sequence modeling tasks.

Meaning Representation (AMR) is a sentence-level meaning representation based on predicate argument structure. One of the challenges we find in AMR parsing is to capture the structure of complex sentences which expresses the relation between predicates. Knowing the core part of the sentence structure in advance may be beneficial in such a task. In this paper, we present a list of dependency patterns for English complex sentence constructions designed for AMR parsing. With a dedicated pattern matcher, all occurrences of complex sentence constructions are retrieved from an input sentence. While some of the subordinators have semantic ambiguities, we deal with this problem through training classification models on data derived from AMR and Wikipedia corpus, establishing a new baseline for future works. The developed complex sentence patterns and the corresponding AMR descriptions will be made public.

As an intersection of artificial intelligence and human communication analysis, Emotion Recognition in Conversation (ERC) has attracted much research attention in recent years. Existing studies, however, are limited in adequately exploiting latent relations among the constituent utterances. In this paper, we address this issue by proposing a novel approach named Latent Relation-Aware Graph Convolutional Network (LR-GCN), where both speaker dependency of the interlocutors is leveraged and latent correlations among the utterances are captured for ERC. Specifically, we first establish a graph model to incorporate the context information and speaker dependency of the conversation. Afterward, the multi-head attention mechanism is introduced to explore the latent correlations among the utterances and generate a set of all-linked graphs. Here, aiming to simultaneously exploit the original modeled speaker dependency and the explored correlation information, we introduce a dense connection layer to capture more structural information of the generated graphs. Through a multi-branch graph network, we achieve a unified representation of each utterance for final prediction. Detailed evaluations on two benchmark datasets demonstrate LR-GCN outperforms the state-of-the-art approaches.

The goal of aspect-based sentiment classification (ASC) is to predict the corresponding emotion of a specific target of a sentence. In neural network-based methods for ASC, various sophisticated models such as Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN) are widespread. Recently, ongoing research has integrated syntactic structures into graph neural networks (GNN) to deal with ASC tasks. However, these methods are limited due to the noise and inefficient use of information of syntactic dependency trees. This paper proposes a novel GNN based deep learning model to overcome the deficiencies of prior studies. In the proposed model, to exploit the information in the syntactic dependency trees, a novel part-of-speech (POS) guided syntactic dependency graph is constructed for a relational graph attention network (RGAT) to eliminate the noises. Further, a syntactic distance attention-guided layer is designed for a densely connected graph convolutional network (DCGCN), which can fully extract semantic dependency between contextual words. Experiments on three public datasets are carried out to evaluate the effectiveness of the proposed model. Comparing to the baselines, our model, as a best alternative, achieves state-of-arts performance.

Industrial processes are becoming increasingly large and complex, thus introducing potential safety risks and requiring an effective approach to maintain safe production. Intelligent process monitoring is critical to prevent losses and avoid casualties in modern industry. As the digitalization of process industry deepens, data-driven methods offer an exciting avenue to address the demands for monitoring complex systems. Nevertheless, many of these methods still suffer from low accuracy and slow response. Besides, most black-box models based on deep learning can only predict the existence of faults, but cannot provide further interpretable analysis, which greatly confines their usage in decision-critical scenarios. In this paper, we propose a novel orthogonal self-attentive variational autoencoder (OSAVA) model for process monitoring, consisting of two components, orthogonal attention (OA) and variational self-attentive autoencoder (VSAE). Specifically, OA is utilized to extract the correlations between different variables and the temporal dependency among different timesteps; VSAE is trained to detect faults through a reconstruction-based method, which employs self-attention mechanisms to comprehensively consider information from all timesteps and enhance detection performance. By jointly leveraging these two models, the OSAVA model can effectively perform fault detection and identification tasks simultaneously and deliver interpretable results. Finally, extensive evaluation on the Tennessee Eastman process (TEP) demonstrates that the proposed OSAVA-based fault detection and identification method shows promising fault detection rate as well as low detection delay and can provide interpretable identification of the abnormal variables, compared with representative statistical methods and state-of-the-art deep learning methods.

Relation extraction between entity pairs is an increasingly critical area in natural language processing. Recently, the pre‐trained bidirectional encoder representation from transformer (BERT) performs excellently on the text classification or sequence labelling tasks. Here, the high‐level syntactic features that consider the dependency between each word and the target entities into the pre‐trained language models are incorporated. Our model also utilizes the intermediate layers of BERT to acquire different levels of semantic information and designs multi‐granularity features for final relation classification. Our model offers a momentous improvement over the published methods for the relation extraction on the widely used data sets.

Feature selection is an important data preprocessing approach that continues to be concerned in data mining . It has been extensively used to construct learning models and reduce storage and computing requirements. Fuzzy rough set is a useful theoretical tool for dealing with the mixed data with fuzziness and inconsistency. Hence, feature selection based on fuzzy rough sets has attracted much attention. However, most of the existing studies ignore the interaction between features, which leads to the loss of useful information. Motivated by this issue, we devise a Dynamic Interaction Feature Selection method based on Fuzzy Rough Set (DIFS_FRS). The method simultaneously considers the interactive relation between features, the relation between conditional features and decision classes, and the dynamic change of feature weights with the variation of feature subset. Firstly, the single-level dependency relevancy between features and classes is defined by the fuzzy dependency degree. Secondly, the multi-level joint interaction between features about classes is investigated. Correspondingly, the correlation evaluation index of features is constructed. Thereafter, a dynamic updating-feedback mechanism is established for a novel feature evaluation function. Finally, compared with the other six representative algorithms on eighteen data sets, the DIFS_FRS algorithm is demonstrated to have better performance.

Time series forecasting has attracted wide attention in recent decades. However, some time series are imbalanced and show different patterns between special and normal periods, leading to the prediction accuracy degradation of special periods. In this paper, we aim to develop a unified model to alleviate the imbalance and thus improving the prediction accuracy for special periods. This task is challenging because of two reasons: (1) the temporal dependency of series, and (2) the tradeoff between mining similar patterns and distinguishing different distributions between different periods. To tackle these issues, we propose a self-attention-based time-varying prediction model with a two-stage training strategy. First, we use an encoder-decoder module with the multi-head self-attention mechanism to extract common patterns of time series. Then, we propose a time-varying optimization module to optimize the results of special periods and eliminate the imbalance. Moreover, we propose reverse distance attention in place of traditional dot attention to highlight the importance of similar historical values to forecast results. Finally, extensive experiments show that our model performs better than other baselines in terms of mean absolute error and mean absolute percentage error.

Received Signal Strength Indicator (RSSI) fingerprinting is known as the most concerned method for indoor localization as its high accuracy and low cost. Numerous RSSI based methods have shown their attractive performances, but the major drawback is the high dependency on the database. In this paper, we propose a multi-floor indoor localization method which includes floor detection and location estimation based on the radial basis function (RBF) network. To ensure the localization accuracy and stability, the network is constructed according to the probabilistic algorithm. Choosing Gaussian radial basis functions with appropriate widths, the network parameters can be initialized appropriately regardless of deficiency of RSSI data. By further conducting the supervised learning of RBF network, the network parameters will be effectively calibrated and updated, so as to achieve a better localization performance. In addition, a radio map quality evaluation criterion is proposed to conduct a comprehensive analysis and interpretation for the localization approach. Finally, experimental results of a publicly accessible dataset which includes multi-floors buildings verify that the performance of the proposed RBF network is superior to other commonly used methods.

Anomaly detection for log sequences is a necessary task for system intelligent operation and fault diagnosis. In a log sequence, adjacent logs have the property of local correlation, while long-distance logs have remote dependencies. It is helpful to fully mine these information during modeling for improving the performance of anomaly detection. Meanwhile, there are some redundant information or noise in the log sequence, which has no contribution to the detection, and may even bring negative impact. The existing methods for log sequence anomaly detection do not take the above problems into account when constructing models. In this paper, we propose LSADNET, an unsupervised log sequence anomaly detection network based on local information extraction and globally sparse Transformer model. LSADNET applies multi-layer convolution to capture the local correlation between adjacent logs, and utilizes Transformer to learn the global dependency among long-distance logs. Meanwhile, we propose a globally sparse Transformer model to improve the self-attention mechanism, which can help to retain important information adaptively and eliminate the irrelevant information in the log sequence. In addition, according to the co-occurrence mode of log templates, we put forward the calculation formula of log template transfer value, and apply it to log vectorization. Through sufficient experiments on two public datasets, it is confirmed that LSADNET has better performance than the state-of-art methods.

Modelling of bridge structures is an important topic due to the increasing need to get accurate information in a visual way. Registration between as-built laser data and global design reference one is necessary to guarantee the validity and dependency of the proposed model. This research presents a new registration method based on artificial neural networks (ANN) methodology. A developed software is implemented to apply ANN and to achieve the research objective. A steel structure bridge is chosen for the application of the proposed system. A point cloud laser data (as built) of the bridge is registered to the design model. The results show that the developed registration method is potential and reliable due to the insignificance of the error between both as-built model and the design model. The assumed level of significance is 95% and the error between as-built model and the design model after the registration process is found to be less than 5%. So, one concludes that the developed method can be used as a registration method for the laser scanning bridge structures. Some recommendations are presented to the future study concerned with using deep learning (DL) to apply the registration process to get better results.

Motivated by a multimodal neuroimaging study for Alzheimer's disease, in this article, we study the inference problem, i.e., hypothesis testing, of sequential mediation analysis. The existing sequential mediation solutions mostly focus on sparse estimation, while hypothesis testing is an utterly different and more challenging problem. Meanwhile, the few mediation testing solutions often ignore the potential dependency among the mediators, or cannot be applied to the sequential problem directly. We propose a statistical inference procedure to test mediation pathways when there are sequentially ordered multiple data modalities and each modality involves multiple mediators. We allow the mediators to be conditionally dependent, and the number of mediators within each modality to diverge with the sample size. We produce the explicit significance quantification and establish the theoretical guarantees in terms of asymptotic size, power, and false discovery control. We demonstrate the efficacy of the method through both simulations and an application to a multimodal neuroimaging pathway analysis of Alzheimer's disease.

Entity typing (ET) is the process of identifying the semantic types of every entity within a corpus. ET involves labelling each entity mention with one or more class labels. As a multi-class, multi-label task, it is considerably more challenging than named entity recognition. This means existing entity typing models require pre-identified mentions and cannot operate directly on plain text. Pipeline-based approaches are therefore used to join a mention extraction model and an entity typing model to process raw text. Another key limiting factor is that these mention-level ET models are trained on fixed context windows, which makes the entity typing results sensitive to window size selection. In light of these drawbacks, we propose an end-to-end entity typing model (E2EET) using a Bi-GRU to remove the dependency on window size. To demonstrate the effectiveness of our E2EET model, we created a stronger baseline mention-level model by incorporating the latest contextualised transformer-based embeddings (BERT). Extensive ablative studies demonstrate the competitiveness and simplicity of our end-to-end model for entity typing.

This article presents a Universal Dependency (UD) annotation scheme for Mandarin Chinese, as well as the current UD Chinese HK treebank. Our focus is mainly on parts-of-speech tags and syntactic relations, with a quite large array of phenomena investigated. The main goal is to make transparent the linguistic consideration behind our annotation choices, and show how we articulated these choices with the criteria of Universal Dependencies. This scheme has been developed with reference to two other dependency schemes for this language, i.e. the Chinese Stanford Dependencies (Chang et al., 2009) and the Chinese Dependency Treebank (HIT-SCIR, 2010). We provide mappings between our scheme and the two others. The content of the UD Chinese HK treebank is discussed in relation to the other UD treebanks for Chinese, and the inter-annotator agreement on POS and dependency annotation is reported. Our proposed scheme is motivated by reasoned linguistic analysis, is suitable for cross-linguistic comparison, and produced a high level of agreement between annotators.

Popular navigation applications and services optimize routes based on either distance or time, disregarding drivers’ preferences when suggesting routes. Various unknown circumstances may affect users’ travel behaviors between two locations on the road network, hence it is complicated to provide satisfactory personalized route recommendations. In this paper, it is believed that users’ travel behaviors are implicitly reflected and can be learned from their historical Global Positioning System (GPS) trajectories. The Behavior-based Route Recommendation (BR2) method is proposed to compute personalized routes based exclusively on users’ travel preferences. The concepts of appearance and transition behaviors are defined to describe users’ travel behaviors. The behaviors are extracted from users’ past travels and the missing behaviors, of unvisited locations, are estimated with the Optimized Random Walk with Restart technique. Furthermore, the temporal dependency of travel behaviors is considered by constructing a time difference interval histogram. A behavior graph is generated to allow the maximum probability route computation with the shortest path algorithm, resulting in the most likely route to be taken by a user. An extension is proposed, named BR2+, to better consider the temporal dependency and incorporate distance in the recommendation process. Experiments conducted on two real GPS trajectory data sets demonstrate the efficiency and effectiveness of the proposed method. In addition, a web-based geographic information system (GIS) called MPR is implemented to demonstrate differences in route recommendation when time, distance, or users’ preferences are considered, besides providing insight about users’ movement through data visualization of their spatial and temporal coverage.

Target-specific sentiment classification has a dependency over the target term extraction. The majority of current studies in sentiment classification tasks do not utilize the complete linguistic and sentiment knowledge. Consequently, strenuous efforts are to be made for expressing the implications of each word from the sentences, which have significant amount of contextual dependencies. Hence, it leads to the problems like loss of semantics, missing of context-dependent information and also results in poor classification of the models. In this paper, we propose a Deep Finesse Network (DFN) to address these limitations and enhance the accuracy. The DFN employs a multichannel paradigm to exploit multi-grained sentiment features by leveraging the existing linguistic and sentiment knowledge more effectively without any human involvement. In each channel, the model firstly extracts the local features from the multi-grained sentiment features and then captures the global and spatial information of the identified local features. Secondly, it directly models the contextual relationships with enriched semantic information from the global features. Subsequently, the intra-sequence relations were also modeled among the contextual features to identify the target features in order to understand and predict the sentiments of identified contextual features. Finally, the effectiveness of the DFN is also evaluated on different datasets. The results proved that DFN outperforms all the current and advanced state-of-art models in classification accuracy in most cases.

Fact extraction, which aims to extract (entity, attribute, value)-tuples from massive text corpora, is crucial in the area of text data mining . Recent approaches have focused on extracting facts by mining textual patterns with semantic types, where the quality of a pattern is evaluated based on content-based criteria, such as frequency. However, these approaches overlook the dimension of pattern reliability , which reflects how likely the extracted facts are correct. As a result, a pattern of good content-quality (e.g., high frequency) may still extract incorrect facts. In this study, we consider both pattern reliability and fact trustworthiness in addressing the pattern-based fact extraction problem. To learn the complex relationship between pattern reliability and fact trustworthiness, we propose a novel deep learning model using a hybrid of the CNN and LSTM architecture. For fact embedding, we adopt CNN to extract a fix-sized representation of each component, i.e., entity, attribute, and value, of the fact. For pattern embedding, we represent the pattern as a semantic composition of its extracted fact representations. To de-emphasis the noisy facts, we consider the fact trustworthiness and frequency during the process of pattern embedding, where the features of the tuple trustworthiness information are extracted by a long short-term memory (LSTM) model. To learn the pattern-fact relational dependency, we train the model with both pattern and tuple labels. Extensive experiments involving three real-world datasets demonstrated that the proposed model significantly improves the quality of the patterns and the extracted facts in the pattern-based information extraction.

Accurately predicting energy consumption (EC) is a difficult task, owing to its inherent complexity and nonlinearity features. To decrease the complexity of EC predictions, applying a novel three-layer decomposition approach to decompose a complex series into a trend sub-series and several simpler non-trend sub-series. Since the trend sub-series describes the developing tendency that is determined by the primarily influencing factors on EC, it is predicted by multivariate linear regression (MLR). In view of its learning ability of nonlinear mappings and excellent performance in dealing with long term dependency, long short-term memory neural network (LSTM) is applied to predict non-trend sub-series. Then, the final EC prediction is solved by calculating the sum of all these sub-series prediction values. A novel EC forecasting model based on machine learning with three-layer decomposition-ensemble approach is proposed in this study. Compared with traditional decomposition approaches (trend decomposition (TD), wavelet decomposition (WD), empirical mode decomposition (EMD), TD-EMD and TD-WD), the proposed three-layer decomposition approach can improve the prediction performance by combining the advantages of TD, EMD and WD. The proposed forecasting model is validated by empirical studies using the EC data of China and U.S. Finally, this study gives the China’s EC prediction in 2020–2024.

Aspect-based sentiment classification (ABSC) is a fine-grained analysis task that obtains different sentiment polarities contained in a single text from the views of different aspects. Its practicability draws so much attention from researchers that the number of related works grows explosively. However, existing works mainly aim to obtain polarities from short texts (shorter than 100 words), only a few works analyze documents (shorter than 500 words), but almost no work analyzes long documents (LD, longer than 500 words). This situation makes ABSC powerless when dealing with some texts like in-depth analysis articles. In this paper, we make ABSC step into the LD level by proposing the Hierarchical Aspect-Oriented Framework for Long Document (HAOFL). HAOFL solves two challenges that rarely appear in short texts and normal documents. The first is the too-long input sequence that can cause the model to forget previously learned information or ignore the tailed unlearned information. The second is the unstable sentiment information of the target aspect contained in LD, which increases the difficulty for a model to draw a proper result. HAOFL constructs the data transformation module, dependency processing module, and sentiment aggregation module to solve these two challenges. Numerical experiments prove HAOFL can solve the aforementioned challenges and achieve superior performance in an effective and resource-saving way. With HAOFL, the performances of popular ABSC models on LD are improved at most 8.69% of accuracy and 11.37% of F1-score. In terms of resource-consuming, up to 82.10% of training time and 71.03% of GPU memory are saved.

Topic models have been widely used for learning the latent explainable representation of documents, but most of the existing approaches discover topics in a flat structure. In this study, we propose an effective hierarchical neural topic model with strong interpretability. Unlike the previous neural topic models, we explicitly model the dependency between layers of a network, and then combine latent variables of different layers to reconstruct documents. Utilizing this network structure, our model can extract a tree-shaped topic hierarchy with low redundancy and good explainability by exploiting dependency matrices. Furthermore, we introduce manifold regularization into the proposed method to improve the robustness of topic modeling. Experiments on real-world datasets validate that our model outperforms other topic models in several widely used metrics with much fewer computation costs.

Exploiting hand-crafted lexicon knowledge to enhance emotional or sentimental features at word-level has become a widely adopted method in emotion-relevant classification studies. However, few attempts have been made to explore the emotion construction in the classification task, which provides insights to how a sentence’s emotion is constructed. The major challenge of exploring emotion construction is that the current studies assume the dataset labels as relatively independent emotions, which overlooks the connections among different emotions. This work aims to understand the coarse-grained emotion construction and their dependency by incorporating fine-grained emotions from domain knowledge. Incorporating domain knowledge and dimensional sentiment lexicons, our previous work proposes a novel method namedEmoChannelto capture the intensity variation of a particular emotion in time series. We utilize the resultant knowledge of 151 available fine-grained emotions to comprise the representation of sentence-level emotion construction. Furthermore, this work explicitly employs a self-attention module to extract the dependency relationship within all emotions and proposeEmoChannel-SANetwork to enhance emotion classification performance. We conducted experiments to demonstrate that the proposed method produces competitive performances against the state-of-the-art baselines on both multi-class datasets and sentiment analysis datasets.

Sleep-wake detection is of great importance for the measurement of sleep quality. In this article, a novel ensemble deep learning framework is proposed to detect sleep-wake states based on heart rate variability (HRV) and acceleration. We firstly design a local feature based long short-term memory (LF-LSTM) network to encode temporal dependency and learn features from acceleration data with high sampling frequency. In the meantime, some handcrafted features are extracted from HRV which has a special data format. After that, we develop a unified framework to integrate these two types of features, i.e., the features extracted from HRV and the features learned by LF-LSTM from acceleration, to form a complete feature set. Finally, an efficient ensemble learning scheme is proposed to further boost the performance of sleep-wake classification. A real dataset has been collected to verify the effectiveness of the proposed approach. We also compare with some well-known benchmark approaches for sleep-wake detection. The results demonstrate that the proposed ensemble deep learning method outperforms all the benchmark approaches.

Aspect level sentiment classification aims to extract fine-grained sentiment expressed towards specific aspects from a sentence. The key to this task lies in connecting aspects and their respective sentiment contexts. Existing methods measure the dependency weights between aspects and context words via either the semantic similarity between words captured by attention mechanism or the structural proximity between words in syntactic structures. However, methods in both groups fail to fully exploit explicit syntactic dependency, which we argue should be critical to identify sentiment contexts. In this paper, we propose a novel syntactic-dependency-based attention network (SDATT) to incorporate explicit syntactic dependency for aspect level sentiment classification. SDATT first models the dependency path between each word and the aspect to characterize aspect-oriented syntactic representation of each word. The generated syntactic representations are later fed into the attention layer to help infer the dependency weights for sentiment prediction. Experimental results on five benchmark datasets show the superior performance of the proposed model over state-of-the-art baselines.

The temporal dependencies of wind power are significant to be involved in the modeling of short-term wind power forecasts. However, different time series inputs will contribute differently to the forecasting performance and bring in challenges to the selection of the relevant driving information. In this paper, a Multi-Source and Temporal Attention Network (MSTAN) is proposed for short-term wind power probabilistic prediction. The MSTAN model introduces multi-source NWP and makes three specific designs to improve prediction performance. Firstly, a novel multi-source variable attention module is proposed to select the driving variables of NWP. Secondly, a temporal attention module is used to capture the implicit temporal dependency hidden in the historical measurements and multi-source NWP sequence. Thirdly, the residual module is wrapped in MSTAN to skip some unnecessary nonlinear transformations and provide adaptive complexity to the entire model. After training, multi-horizon density forecasts for the next 48 hours are yielded by MSTAN. The MSTAN is compared with state-of-the-art machine learning schemes in the wind power forecasting system using the operation data from 3 wind farms. We demonstrate that MSTAN outperforms other counterparts on both deterministic and probabilistic prediction. The structure design scheme of MSTAN has been proven effective.

Long short-term memory (LSTM) or gated recurrent units (GRUs) are usually employed to recurrently learn variable-length sentence representations with long-range dependency in document-level sentiment analysis. However, LSTM and GRUs are biased models in which the words in the tail of sentences are dominant over the words at the beginning. Although some reweighting methods, such as self-attention, are useful for reweighting each word according to its importance, when processing a document with many tokens, they tend to assign equally smaller weights to each word, leading to the keywords being covered by nonsentiment words. In this paper, a recurrent attention LSTM neural network is presented to iteratively locate an attention region covering the key sentiment words. By gradually reducing the range of attention and the number of tokens, the model can leverage the weight of the key sentiment words for final classification. Additionally, a joint loss function is implemented to highlight both keywords and appropriate attention regions. The comparative experiments are conducted on the IMDB, Yelp and Amazon document-level corpora. The results show that the proposed model outperforms several state-of-the-art methods in document-level sentiment classification.

The more knowledge industrial practitioners detain of their production processes, the more they are capable of performing process improvements. Nonetheless, there may exist process characteristics and dependencies that are not easily extractable from business models, such as routing dependent attributes. This paper introduces an algorithm-driven framework to establish whether process path decisions influence the attributes in non-direct sequences, e.g., deploying machine A instead of machine B affects the % of rejected parts on the process, 4 stages down the line. This problem is shown to bears similarities with sequential pattern mining problems. The basis of the solution framework relies on process mining and data mining techniques. The approach proposed is applied on a real industrial log, unveiling deficiencies in the system and providing further improvement recommendations.

Most common methods for automatic text analysis in communication science ignore syntactic information, focusing on the occurrence and co-occurrence of individual words, and sometimes n-grams. This is remarkably effective for some purposes, but poses a limitation for fine-grained analyses into semantic relations such as who does what to whom and according to what source. One tested, effective method for moving beyond this bag-of-words assumption is to use a rule-based approach for labeling and extracting syntactic patterns in dependency trees. Although this method can be used for a variety of purposes, its application is hindered by the lack of dedicated and accessible tools. In this paper we introduce the rsyntax R package, which is designed to make working with dependency trees easier and more intuitive for R users, and provides a framework for combining multiple rules for reliably extracting useful semantic relations.

Groundwater dependency is increasing globally, while millions of potentially contaminated sites are yet to be characterized for contamination levels. In particular, groundwater contamination due to light nonaqueous phase liquids (LNAPLs) continues to be a global challenge. Mathematical approaches (i.e., analytical, semi-analytical, empirical, numerical) are preferred for an initial site assessment to circumvent the high characterization costs and limited site data availability. However, the site-specific nature of contamination restricts the generalization of any single approach. Hence, the requirement is for an easy-to-use computing interface that provides site-specific data management, the selection and use of multiple-model interfaces for computing, and site characterization, with extension for the latest models as they become available. This work provides one such interface called CAST or Contamination Assessment and Site-management Tool. CAST is an open-source browser-based (online/offline) tool that provides an interface for six different analytical models (e.g., BIOSCREEN-AT), a MODFLOW based numerical model, and two empirical models (including a hybrid numerical-analytical model). Additionally, CAST includes interfaces for site data management, their evaluation, and scenario-based modeling. CAST's development is in a modular format, which simplifies the addition of new computing or data interfaces. Furthermore, the entire code-base of CAST is based on open-source (dominantly Python programming) libraries and standards. This further simplifies the modification or extension of this tool. This paper introduces CAST, its different computing, and data management interfaces and provides examples of the tool's functionality primarily for the initial evaluation of contaminated sites.

For the classification task of machine learning algorithms independency between conditional attributes is a precondition for success of data mining. On the other hand, decision trees are one of the mostly used machine learning algorithms because of their good understandability. So, because dependency between conditional attributes can cause more complex trees, supplying conditional attributes independent each other is very important, the requirement of conditional attributes for decision trees as well as other machine learning algorithms is that they are independent each other and dependent on decisional attributes only. Statistical method to check independence between attributes is Chi-square test, but the test can be effective for categorical attributes only. So, the applicability of Chi-square test is limited, because most datasets for data mining have mixed attributes of categorical and numerical. In order to overcome the problem, and as a way to test dependency between conditional attributes, a novel method based on functional dependency based on data that can be applied to any datasets irrespective of data type of attributes is suggested. After removing highly dependent attributes between conditional attributes, we can generate better decision trees. Experiments were performed to show that the method is effective, and the experiments showed very good results.

Money laundering is an act of criminals attempting to cover up the nature and source of their illegal gains. Large-scale money laundering has a great harm to a country’s economy, political order and even social stability. Therefore, it is essential to predict the risk of money laundering scientifically and reasonably. Money laundering data have complex temporal dependency. Historical transactions have an impact on current transactions. Different transactions also have complex spatial correlation. For this very reason, a hybrid spatiotemporal money laundering prediction model based on graph convolution neural networks (GCN) and long short-term memory (LSTM), abbreviated MGC-LSTM, is proposed to learn the dependency between different money laundering transactions. Firstly, LSTM is employed to obtain the temporal dependence of money laundering data set at different times; secondly, GCN is wielded to learn the complex spatial dependency of different money laundering transactions. Historical observations on different transactions, temporal and transactions features are defined as graph signals. For each time stamp, the results trained by LSTM are served as the input of GCN; finally, we compare the MGC-LSTM with other state-of-the-art algorithms to evaluate the performance of the proposed method. The experimental results demonstrate that MGC-LSTM outperforms other comparing algorithms with respect to effectiveness and significance.