Search

Multimodal aspect-based sentiment analysis (MABSA) aims to determine the sentiment polarity of each aspect mentioned in the text based on multimodal content. Various approaches have been proposed to model multimodal sentiment features for each aspect via modal interactions. However, most existing approaches have two shortcomings: (1) The representation gap between textual and visual modalities may increase the risk of misalignment in modal interactions; (2) In some examples where the image is not related to the text, the visual information may not enrich the textual modality when learning aspect-based sentiment features. In such cases, blindly leveraging visual information may introduce noises in reasoning the aspect-based sentiment expressions. To tackle these shortcomings, we propose an end-to-end MABSA framework with image conversion and noise filtration. Specifically, to bridge the representation gap in different modalities, we attempt to translate images into the input space of a pre-trained language model (PLM). To this end, we develop an image-to-text conversion module that can convert an image to an implicit sequence of token embedding. Moreover, an aspect-oriented filtration module is devised to alleviate the noise in the implicit token embeddings, which consists of two attention operations. After filtering the noise, we leverage a PLM to encode the text, aspect, and image prompt derived from filtered implicit token embeddings as sentiment features to perform aspect-based sentiment prediction. Experimental results on two MABSA datasets show that our framework achieves state-of-the-art performance. Furthermore, extensive experimental analysis demonstrates the proposed framework has superior robustness and efficiency. [ABSTRACT FROM AUTHOR]

Emotion recognition in conversations (ERC) needs to detect the emotion of each utterance in conversations. However, it is difficult for machines to recognize the emotion of utterances like humans, partly because of the lack of commonsense knowledge. Despite existing efforts gradually incorporate knowledge in ERC, they can not adaptively adjust knowledge according to different utterances and their context. In this article, we propose a knowledge selection framework SKSEC (Select Knowledge in light of Sentiment Emotion and Context). In the SKSEC framework, first, external knowledge is eliminated by three Knowledge Elimination (KE) modules. More concretely, In word-level KE, the concept knowledge different from the sentiment corresponding to the word in utterances is randomly eliminated. In utterance- or context-level KE, If the similarity between the knowledge representation and the emotion label representation of the current utterance or its context is less than the preset threshold, the knowledge will be eliminated. Then we refine the weight of knowledge using two Graph ATtention (GAT) mechanisms. Specifically, In Sentics GAT, we employ a dimensional emotion model to measure words in utterances and their corresponding knowledge and adjust the weight of knowledge according to their emotional similarity. In Semantics GAT, the weight of knowledge is adjusted according to the semantic similarity between context and incorporated knowledge. Finally, we feed the selected knowledge to the most advanced models to evaluate the quality of knowledge. The experimental results show that the SKSEC framework can effectively improve the performance of the model by eliminating and refining external knowledge in different size and domain datasets. [ABSTRACT FROM AUTHOR]

The state-of-the-art approaches to targeted aspect-based sentiment analysis (TABSA) are mostly built on deep neural networks with attention mechanisms. One problem is that embeddings of targets and aspects are either pre-trained from large external corpora or randomly initialized. We argue that affective commonsense knowledge and words indicative of sentiment could be used to learn better target and aspect embeddings. We therefore propose an embedding refinement framework called RAEC (Refining Affective Embedding from Context), in which sentiment concepts extracted from affective commonsense knowledge and word relative location information are incorporated to derive context-affective embeddings. Furthermore, a sparse coefficient vector is exploited in refining the embeddings of targets and aspects separately. In this way, embeddings of targets and aspects can capture the highly relevant affective words. Experimental results on two benchmark datasets show that our framework can be easily integrated with existing embedding-based TABSA models and achieves state-of-the-art results compared to models relying on pre-trained word embeddings or built on other embedding refinement methods. [ABSTRACT FROM AUTHOR]

Recently, advances in neural network approaches have achieved many successes in both sentiment classification and probabilistic topic modeling. On the one hand, latent topics derived from the global context of documents could be helpful in capturing more accurate word semantics and hence could potentially improve the sentiment classification accuracy. On the other hand, the word-level attention vectors obtained during the learning of sentiment classifiers could carry word-level polarity information and can be used to guide the discovery of topics in topic modeling. This paper proposes a multi-task learning framework which jointly learns a sentiment classifier and a topic model by making the word-level latent topic distributions in the topic model to be similar to the word-level attention vectors in sentiment classifiers through mutual learning. Experimental results on the Yelp and IMDB datasets verify the superior performance of the proposed framework over strong baselines on both sentiment classification and topic modeling. The proposed framework also extracts more interpretable topics compared to other conventional topic models and neural topic models. [ABSTRACT FROM AUTHOR]

This article copes with vibration suppression and angular position tracking problems of a robotic manipulator system comprised of a rotating hub and a length-varying manipulator. To obtain precise dynamic response, the manipulator system is modeled in infinite-dimension with partial differential equations. S-curve acceleration/deceleration (S-CA/D) scheme is employed for speed regulation of the length-varying manipulator. Two novel observers are developed to estimate both the unknown disturbances and their time-derivatives, and two auxiliary systems are put forward to tackle input constraints. With assistance of the auxiliary systems and observers, two boundary control laws are put forward to manage vibration suppression and angular position tracking of the proposed manipulator system. Through Lyapunov’s theory, the closed-loop system is proved to be bounded. Numerical simulations have displayed the effectiveness of the observers and boundary control laws. [ABSTRACT FROM AUTHOR]

Cross-modal image-text retrieval has emerged as a challenging task that requires the multimedia system to bridge the heterogeneity gap between different modalities. In this paper, we take full advantage of image-to-text and text-to-image generation models to improve the performance of the cross-modal image-text retrieval model by incorporating the text-grounded and image-grounded generative features into the cross-modal common space with a “Two-Teacher One-Student” learning framework. In addition, a dual regularizer network is designed to distinguish the mismatched image-text pairs from the matched ones. In this way, we can capture the fine-grained correspondence between modalities and distinguish the best-retrieved result from a candidate set. Extensive experiments on three benchmark datasets (i.e., MIRFLICKR-25K, NUS-WIDE, and MS COCO) show that our model can achieve state-of-the-art cross-modal retrieval results. In particular, our model improves the image-to-text and text-to-image retrieval accuracy by more than 22% over the best competitors on the MS COCO dataset. [ABSTRACT FROM AUTHOR]

Stance classification aims at identifying, in the text, the attitude toward the given targets as favorable, negative, or unrelated. In existing models for stance classification, only textual representation is leveraged, while commonsense knowledge is ignored. In order to better incorporate commonsense knowledge into stance classification, we propose a novel model named commonsense knowledge enhanced memory network, which jointly represents textual and commonsense knowledge representation of given target and text. The textual memory module in our model treats the textual representation as memory vectors, and uses attention mechanism to embody the important parts. For commonsense knowledge memory module, we jointly leverage the entity and relation embeddings learned by TransE model to take full advantage of constraints of the knowledge graph. Experimental results on the SemEval dataset show that the combination of the commonsense knowledge memory and textual memory can improve stance classification. [ABSTRACT FROM AUTHOR]

Most proposed methods for TF-binding site (TFBS) predictions only use low order dependencies for predictions due to the lack of efficient methods to extract higher order dependencies. In this work, we first propose a novel method to extract higher order dependencies by applying CNN on histone modification features. We then propose a novel TFBS prediction method, referred to as CNN_TF, by incorporating low order and higher order dependencies. CNN_TF is first evaluated on 13 TFs in the mES cell. Results show that using higher order dependencies outperforms low order dependencies significantly on 11 TFs. This indicates that higher order dependencies are indeed more effective for TFBS predictions than low order dependencies. Further experiments show that using both low order dependencies and higher order dependencies improves performance significantly on 12 TFs, indicating the two dependency types are complementary. To evaluate the influence of cell-types on prediction performances, CNN_TF was applied to five TFs in five cell-types of humans. Even though low order dependencies and higher order dependencies show different contributions in different cell-types, they are always complementary in predictions. When comparing to several state-of-the-art methods, CNN_TF outperforms them by at least 5.3 percent in AUPR. [ABSTRACT FROM AUTHOR]

Most past works for DNA-binding residue prediction did not consider the relationships between residues. In this paper, we propose a novel approach for DNA-binding residue prediction, referred to as EL_LSTM, which includes two main components. The first component is the Long Short-Term Memory (LSTM), which learns pairwise relationships between residues through a bi-gram model and then learns feature vectors for all residues. The second component is an ensemble learning based classifier introduced to tackle the data imbalance problem in binding residue predictions. We use a variant of the bagging strategy in ensemble learning to achieve balanced samples. Evaluations on PDNA-224 and DBP-123 show that adding feature relationships performs better than classifiers without feature relationships by at least 0.028 on MCC, 1.18 percent on ST and 0.012 on AUC. This indicates the usefulness of feature relationships for DNA-binding residue predictions. Evaluation on using ensemble learning indicates that the improvement can reach at least 0.021 on MCC, 1.32 percent on ST, and 0.018 on AUC compared to the use of a single LSTM classifier. Comparisons with the state-of-the-art predictors show that our proposed EL_LSTM outperforms them significantly. Further feature analysis validates the effectiveness of LSTM for the prediction of DNA-binding residues. [ABSTRACT FROM AUTHOR]

In product reviews, it is observed that the distribution of polarity ratings over reviews written by different users or evaluated based on different products are often skewed in the real world. As such, incorporating user and product information would be helpful for the task of sentiment classification of reviews. However, existing approaches ignored the temporal nature of reviews posted by the same user or evaluated on the same product. We argue that the temporal relations of reviews might be potentially useful for learning user and product embedding and thus propose employing a sequence model to embed these temporal relations into user and product representations so as to improve the performance of document-level sentiment analysis. Specifically, we first learn a distributed representation of each review by a one-dimensional convolutional neural network. Then, taking these representations as pretrained vectors, we use a recurrent neural network with gated recurrent units to learn distributed representations of users and products. Finally, we feed the user, product and review representations into a machine learning classifier for sentiment classification. Our approach has been evaluated on three large-scale review datasets from the IMDB and Yelp. Experimental results show that: (1) sequence modeling for the purposes of distributed user and product representation learning can improve the performance of document-level sentiment classification; (2) the proposed approach achieves state-of-the-art results on these benchmark datasets. [ABSTRACT FROM PUBLISHER]

The article reports that the raw size of a high-dynamic range (HDR) image brings about problems in storage and transmission. Many bytes are wasted in data redundancy and perceptually unimportant information. To address this problem, researchers have proposed some preliminary algorithms to compress the data, like RGBE/XYZE, OpenEXR, LogLuv and so on. These algorithms mostly use lossless compression strategies, so they are not capable of providing significant compression. HDR images can have a dynamic range of more than four orders of magnitude while conventional 8-bit images retain only two orders of magnitude of the dynamic range. This distinction between an HDR image and a conventional image leads to difficulties in using most existing image compressors. Joint Photographic Experts Group (JPEG) 2000 supports up to 16-bit integer data, so it can already provide image compression for most HDR images. In this article the researchers propose a JPEG 2000-based lossy image compression scheme for HDR images of all dynamic ranges. They show how to fit HDR encoding into a JPEG 2000 encoder to meet the HDR encoding requirement. To achieve the goal of minimum error in the logarithm domain, they map the logarithm of each pixel value into integer values and then send the results to a JPEG 2000 encoder. Their approach is basically a wave let-based HDR still-image encoding method.

In this article the authors present a new local adaptive noise reduction kernel. This new operator suppresses the outliers and inter-pixel incoherence in a non-iterative way. Computing accurate global illumination is one of the most challenging tasks in computer graphics. Its difficulty comes from the complexity of the rendering equation. Monte Carlo is a major method used to solve this equation. Although it's powerful enough to compute all global illumination effects, it requires too much rendering time to accurately render a typical scene. Limiting the rendering time often brings about a particular artifact in the final rendered image known as Monte Carlo noise. An alternative way to render images using Monte Carlo rendering is to use a low sampling density and then de-noise the image in a post-processing stage. Although a classic image noise reduction algorithm can be used to reduce this particular type of noise, a naive application is not always effective. So, researchers have developed various methods for effective and efficient post-processing of Monte Carlo noise.