Search

Abstract Aspect-level sentiment classification, as a fine-grained task in sentiment classification, aiming to extract sentiment polarity from opinions towards a specific aspect word, has been made tremendous improvements in recent years. There are three key factors for aspect-level sentiment classification: contextual semantic information towards aspect words, correlations between aspect words and their context words, and location information of context words with regard to aspect words. In this paper, two models named AE-DLSTMs (Attention-Enabled Double LSTMs) and AELA-DLSTMs (Attention-Enabled and Location-Aware Double LSTMs) are proposed for aspect-level sentiment classification. AE-DLSTMs takes full advantage of the DLSTMs (Double LSTMs) which can capture the contextual semantic information in both forward and backward directions towards aspect words. Meanwhile, a novel attention weights generating method that combines aspect words with their contextual semantic information is designed so that those weights can make better use of the correlations between aspect words and their context words. Besides, we observe that context words with different distances or different directions towards aspect words have different contributions in sentiment polarity. Based on AE-DLSTMs, the location information of context words by assigning different weights is incorporated in AELA-DLSTMs to improve the accuracy. Experiments are conducted on two English datasets and one Chinese dataset. The experimental results have confirmed that our models can make remarkable improvements and outperform all the baseline models in all datasets, improving the accuracy of 1.67 percent to 4.77 percent in different datasets compared with baseline models. 1 1 Our code is open-source and available at https://github.com/rxt2012kc/AELA-DLSTMs. [ABSTRACT FROM AUTHOR]

Highlights • We propose an embedding model for popularity prediction, include time embedding and joint embedding of users and words. • To focus on the informative parts and suppress noisy ones, our proposed model also introduces attention mechanism. • We propose three encoders by considering text content, user and popularity time series information for social media content. • We use large social media datasets to show the effectiveness of our neural popularity prediction (NPP) model. Abstract Online interactive behaviors between Web users often make some social media contents go viral. The popularity of social media contents can help us understand public interest and attention behind user interactions, thus popularity prediction of online contents has become a key task in social media analytics and can facilitate many applications in different domains. However, it is a difficult task for two main reasons. Firstly, popularity can be affected by many factors such as user, text content and time. Secondly, social media data is often noisy, which may degrade the performance of the prediction model. To overcome these difficulties, in this paper, we design a deep learning based popularity prediction model, which extracts and fuses the rich information of text content, user and time series in a data-driven fashion. To deal with the noise in social media data, we incorporate attention mechanism to focus on more informative parts and suppress noisy ones. Experiments on real world datasets demonstrate the effectiveness of our proposed model. [ABSTRACT FROM AUTHOR]

Abstract Recurrent Neural Network (RNN) based approaches have recently shown promising in tackling Natural Language Generation (NLG) problems. This paper presents an approach to leverage gating mechanisms, in which we incrementally propose three additional semantic cells into a traditional RNN model: a Refinement cell to filter the sequential inputs before RNN computations, an Adjustment cell, and an Output cell to select semantic elements and gate a feature vector during generation. The proposed gating-based generators can learn from unaligned data by jointly training both sentence planning and surface realization to generate natural language utterances. We conducted extensive experiments on four different NLG domains in which the results empirically show that the proposed methods not only achieved better performance on all the NLG domains in comparison with previous gating-based, attention-based methods, but also obtained highly competitive results compared to a hybrid generator. [ABSTRACT FROM AUTHOR]

Abstract Attention based encoder-decoder models have shown a great success on video captioning. Recent multi-modal video captioning mainly focused on applying the attention mechanism to all modalities and fusing them in the same level. However, the connections among specific modalities have not been investigated in the fusion process. In this paper, the expressivity of uni-modal is firstly investigated. Due to the characteristic of attention mechanism, an instance-level of visual content is exploited to refine the temporal features. Then, a semantic detection architecture based on CNN+RNN is also employed on the spatiotemporal content to exploit the correlations between semantic labels for better video semantic representation. Finally, a hierarchical attention-based multimodal fusion model for video captioning is proposed by jointly considering the intrinsic properties of multimodal features. Experimental results on the MSVD and MSR-VTT datasets show that the proposed method has achieved competitive performance compared with the related video captioning methods. [ABSTRACT FROM AUTHOR]

Heterogeneous information networks (HIN) based models are now widely used to fuse auxiliary information for personalized recommendation. However, existing works mainly focus on capturing high-order connections between heterogeneous nodes through predefined pattern (such as meta-path, etc.), which depends on relevant domain knowledge and ignores the statistical characteristics of the substructures in HIN. How to capture meaningful personalized behavior patterns, especially the selection patterns (i.e. how a user select an item), and incorporate them into the preference model is still a challenge in the research of HIN-based recommendation. Therefore, in this paper, a specifically-designed Colored-Motif Attention Network (CMoAN) is proposed to deal with this problem. In the proposed CMoAN, colored motif are used as the elemental building blocks to represent context-based selection patterns, and then by constructing a motif-based adjacency matrix to capture higher-order semantic association between nodes in HIN. Besides, an attentive graph neural network is designed to efficiently model the semantics and high-order relations information. Extensive experiments on three real-world datasets demonstrate that CMoAN consistently outperforms state-of-the-art methods. Furthermore, experimental results also verify the effectiveness of using colored motif for capturing users'selection patterns. [ABSTRACT FROM AUTHOR]

Long short-term memory (LSTM) is an extension of the recurrent neural network (RNN) and has achieved excellent performance in various tasks, especially sequential problems. The LSTM is chain-structured, and its architecture is limited by sequential information propagation. In practice, it is hard to solve the problems involving very long term dependencies. Recent studies have indicated that adding recurrent skip connections across multiple timescales may help the RNN improve its performance in long-term dependencies. Moreover, capturing local features can improve the performance of the RNN. In this paper, we propose a novel architecture (Att-LSTM) for the LSTM, which connects continuous hidden states of previous time steps to the current time step and applies an attention mechanism to these hidden states. This architecture can not only capture local features effectively but also help learn very long-distance correlations in an input sequence. We evaluate Att-LSTM in various sequential tasks, such as adding problem, sequence classification, and character-level language modeling. In addition, to prove the generalization and practicality of the novel architecture, we design a character-level hierarchical Att-LSTM and refine the word representation with a highway network. This hierarchical model achieved excellent performance on question classification. [ABSTRACT FROM AUTHOR]

Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in many language pairs. While there are variants of the attention mechanism, all of them use only temporal attention where one scalar value is assigned to one context vector corresponding to a source word. In this paper, we propose a fine-grained (or 2D) attention mechanism where each dimension of a context vector will receive a separate attention score. In experiments with the task of En-De and En-Fi translation, the fine-grained attention method improves the translation quality in terms of BLEU score. In addition, our alignment analysis reveals how the fine-grained attention mechanism exploits the internal structure of context vectors. [ABSTRACT FROM AUTHOR]

Word embeddings and CNN (convolutional neural networks) architecture are crucial ingredients of sentiment analysis. However, sentiment and lexicon embeddings are rarely used and CNN is incompetent to capture global features of sentence. To this end, semantic embeddings, sentiment embeddings and lexicon embeddings are applied for texts encoding, and three different attentions including attention vector, LSTM (long short term memory) attention and attentive pooling are integrated with CNN model in this paper. Additionally, a word and its context are explored to disambiguate the meaning of the word for rich input representation. To improve the performance of three different attention CNN models, CCR (cross-modality consistent regression) and transfer learning are presented. It is worth noticing that CCR and transfer learning are used in textual sentiment analysis for the first time. Finally, some experiments on two different datasets demonstrate that the proposed attention CNN models achieve the best or the next-best results against the existing state-of-the-art models. [ABSTRACT FROM AUTHOR]