Search

Aspect-based sentiment analysis is an important task in fine-grained sentiment analysis, which aims to infer the sentiment towards a given aspect. Previous studies have shown notable success when sufficient labeled training data is available. However, annotating adequate data is labor-intensive, which sets substantial barriers for generalizing the sentiment predictor to the new domain. Two main challenges exist in cross-domain aspect-based sentiment analysis. One challenge is acquiring the domain-invariant knowledge; the other challenge is mining the syntactic-related words towards the aspect-term. In this article, we propose a transformer-based semantic-primary knowledge transferring network (TSPKT) for cross-domain aspect-term sentiment analysis, which utilizes semantic-primary knowledge as a bridge to enable knowledge transfer across domains. Specifically, we first build an S-Graph from external semantic lexicons, and extract the semantic-primary knowledge from the S-Graph. Second, AoaGraphormer is proposed to learn the syntactically relevant words towards the aspect-term. Third, we extend the standard biLSTM classifier to fully integrate the semantic-primary knowledge by adding a novel knowledge-aware memory unit (KAMU) to the biLSTM cell. Extensive experiments on six cross-domain setups demonstrate the superiority of TSPKT against the state-of-the-art baseline methods. [ABSTRACT FROM AUTHOR]

Satellite image sequence prediction is a challenging and significant task. The existing deep learning methods for the task make predictions mainly based on low-level pixelwise features, which fail to model the sophisticated spatial–temporal features of satellite image sequences and deliver unsatisfactory performance. In this letter, we present a hierarchical spatial–temporal network (HSTnet) for satellite image sequence prediction. With a carefully designed hierarchical feature extraction mechanism, HSTnet can learn effective spatial–temporal features from both pixel level and patch level. In addition, to better capture patch-level spatial–temporal dynamics, a dual-branch Transformer is proposed to model patch-level spatial and temporal features, respectively. Comprehensive experiments on the Fengyun-4A (FY-4A) satellite dataset demonstrate the superiority and effectiveness of our proposed method HSTnet over state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Few-shot remote sensing scene classification (FSRSSC) is an important task, which aims to recognize novel scene classes with few examples. Recently, several studies attempt to address the FSRSSC problem by following few-shot natural image classification methods. These existing methods have made promising progress and achieved superior performance. However, they all overlook two unique characteristics of remote sensing images: 1) object cooccurrence that multiple objects tend to appear together in a scene image and 2) object spatial correlation that these cooccurrence objects are distributed in the scene image following some spatial structure patterns. Such unique characteristics are very beneficial for FSRSSC, which can effectively alleviate the scarcity issue of labeled remote sensing images since they can provide more refined descriptions for each scene class. To fully exploit these characteristics, we propose a novel scene-graph-matching-based meta-learning framework for FSRSSC, called SGMNet. In this framework, a scene graph construction module is carefully designed to represent each test remote sensing image or each scene class as a scene graph, where the nodes reflect these cooccurrence objects; meanwhile, the edges capture the spatial correlations between these cooccurrence objects. Then, a scene graph matching module is further developed to evaluate the similarity score between each test remote sensing image and each scene class. Finally, based on the similarity scores, we perform the scene class prediction via a nearest neighbor classifier. We conduct extensive experiments on the UCMercedLandUse, WHU19, AID, and NWPU-RESISC45 datasets. The experimental results show that our method obtains superior performance over the previous state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Satellite image sequence prediction is a crucial and challenging task. Previous studies leverage optical flow methods or existing deep learning methods on spatial–temporal sequence models for the task. However, they suffer from either oversimplified model assumptions or blurry predictions and sequential error accumulation issue, for a long-term forecast requirement. In this article, we propose a novel multiscale time conditional generative adversarial network (MSTCGAN). To address the sequential error accumulation issue, MSTCGAN adopts a parallel prediction framework to produce the future image sequences by a one-hot time condition input. In addition, a powerful multiscale generator is designed with the multihead axial attention, which helps to carefully preserve the fine-grained details for appearance consistency. Moreover, we develop a temporal discriminator to address the blurry issue and maintain the motion consistency in prediction. Extensive experiments have been conducted on the FengYun-4A satellite dataset, and the results demonstrate the effectiveness and superiority of the proposed method over state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting is an important task, which can be used in numerous applications. The key challenge of the task lies in radar echo map prediction. Previous studies leverage the convolutional recurrent neural network (ConvRNN) to address the problem. However, the approaches are built upon mean square losses, and the results tend to have inaccurate appearances, shapes, and positions for predictions. To alleviate this problem, we explore the idea of adversarial regularization and systematically compare four types of generative adversarial networks (GANs), which are the combinations of GAN/Wasserstein GAN (WGAN) and its multiscale version. Extensive experiments on a real-world radar dataset and four typical meteorological examples are conducted. The results validate the effectiveness of adversarial regularization. The developed models show superior performances over the existing prediction approaches in the majority of circumstances. Moreover, we find that the WGAN regularization often delivers better results than the GAN regularization due to its robustness, and the multiscale WGAN, in general, performs the best among all the methods. To reproduce the results, we release the source code at https://github.com/luochuyao/MultiScaleGAN and the test system at http://39.97.217.145:80/. [ABSTRACT FROM AUTHOR]

Cloud detection is the task of detecting cloud areas in remote sensing images (RSIs), and it has attracted extensive research interest. Recently, deep learning (DL)-based methods have been proposed and achieved great performance for cloud detection. However, due to the satellite’s limitation in storage and memory, existing DL approaches, which suffer from extensive computation and large model size, are almost impossible to be deployed on satellites. To fill this gap, we target at studying effective and efficient cloud detection solutions that are suitable for satellites. In this article, we develop a lightweight autoencoder-based cloud detection method, namely, lightweight encoder–decoder cloud detection network (LWCDnet). In the encoder part, the designed novel lightweight dual-branch block (LWDBB) in the backbone extracts spatial and contextual information concurrently. Moreover, a lightweight feature pyramid module (LWFPM) is proposed to capture high-level multiscale contextual information. In the decoder part, the lightweight feature fusion module (LWFFM) compensates for the missing spatial and detail information from the encoder to the high-level feature maps. We evaluate the proposed method on two public datasets: LandSat8 and Moderate-Resolution Imaging Spectroradiometer (MODIS). Extensive experiments demonstrate that the proposed LWCDnet achieves comparable accuracy as the state-of-the-art cloud detection methods and lightweight semantic segmentation algorithms. In the meantime, LWCDnet has much less computation burden with smaller model size. [ABSTRACT FROM AUTHOR]

Cross-modal hashing is an important tool for retrieving useful information from very-high-resolution (VHR) optical images and synthetic aperture radar (SAR) images. Dealing with the intermodal discrepancies, including both spatial–spectral and visual semantic aspects, between VHR and SAR images is extremely vital to generate high-quality common hash codes in the Hamming space. However, existing cross-modal hashing methods ignore the spatial–spectral discrepancy when representing VHR and SAR images. Moreover, existing methods employ derived supervised signals, such as pairwise training images, to implicitly guide hashing learning, which fails to effectively deal with the visual semantic discrepancy, i.e., cannot adequately preserve the intraclass similarity and interclass discrimination between VHR and SAR images. To address these drawbacks, this article proposes a multisensor fusion and explicit semantic preserving-based deep Hashing method, termed as MsEspH, which can effectively deal with the discrepancies. Specifically, we design a novel cross-modal hashing network to eliminate the spatial–spectral discrepancies by fusing extra multispectral images (MSIs), which are generated in real time by a generative adversarial network. Then, we propose an explicit semantic preserving-based objective function by analyzing the connection between classification and hash learning. The objective function can preserve the intraclass similarity and interclass discrimination with class labels directly. Moreover, we theoretically verify that hash learning and classification can be unified into a learning framework under certain conditions. To evaluate our method, we construct and release a large-scale VHR-SAR image dataset. Extensive experiments on the dataset demonstrate that our method outperforms various state-of-the-art cross-modal hashing methods. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting is an important task in weather forecast. The key challenge of the task lies at radar echo map extrapolation. Recent studies show that a convolutional recurrent neural network (ConvRNN) is a promising direction to solve the problem. However, the extrapolation results of the existing ConvRNN methods tend to be blurring and unrealistic. Recent studies show that generative adversarial network (GAN) is a promising tool to address the drawback, while it suffers from the instability for training. In this letter, we build a novel ConvRNN model based on the energy-based GAN for radar echo map extrapolation. The method can alleviate the blurring and unrealistic issues and is more stable. We have conducted experiments on a real-world data set, and the results show that the proposed method outperforms several existing models, including optical flow, convolution gated recurrent unit (ConvGRU), and generative adversarial ConvGRU (GA-ConvGRU). [ABSTRACT FROM AUTHOR]

Poverty prediction is an important task for developing countries that lack the key measures of economic development. The prediction can help governments to allocate scarce resources for sustainable development. Nighttime satellite imagery offers an opportunity to address the task. However, as the nighttime satellite data contain a large amount of noise, directly leveraging it is not very effective. Previous studies have shown that relying on deep learning techniques nighttime satellite data can be a good proxy between daytime satellite imagery and the poverty index. In this letter, based on the proxy, we leverage four deep learning approaches, namely, VGG-Net, Inception-Net, ResNet, and DenseNet, to extract deep features from daytime satellite imagery and then apply least absolute shrinkage and selection operator (LASSO) regression for poverty prediction. To further enhance the performance, we also integrate the squeeze and excitation (SE) module and focal loss into ResNet and DenseNet. Experimental results demonstrate the effectiveness of the investigated approaches, and the DenseNet with SE module and focal loss performs the best. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting is an important task, which can serve numerous applications such as urban alert and transportation. Previous studies leverage convolutional recurrent neural networks (RNNs) to address the problem. However, they all suffer from two inherent drawbacks of the convolutional RNN, namely, the lack of a memory cell to preserve the fine-grained spatial appearances and the position misalignment issue when combining current observations with previous hidden states. In this article, we aim to overcome the defects. Specifically, we propose a novel pseudo flow spatiotemporal LSTM unit (PFST-LSTM), where a spatial memory cell and a position alignment module are developed and embedded in the structure of LSTM. Upon the PFST-LSTM units, we develop a new sequence-to-sequence architecture for precipitation nowcasting, which can effectively combine the spatial appearances and motion information. Extensive empirical evaluations are conducted on synthetic MovingMNIST++ and CIKM AnalytiCup 2017 datasets. Our experimental results demonstrate the superiority of the proposed PFST-LSTM over the state-of-the-art competitors. To reproduce the results, we release the source code at: https://github.com/luochuyao/PFST-LSTM. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting is an important task in operational weather forecasts. The key challenge of the task is the radar echo map extrapolation. The problem is mainly solved by an optical-flow method in existing systems. However, the method cannot model rapid and nonlinear movements. Recently, a convolutional gated recurrent unit (ConvGRU) method is developed, which aims to model such movements based on deep learning techniques. Despite the promising performance, ConvGRU tends to yield blurring extrapolation images and fails to multi-modal and skewed intensity distribution. To overcome the limitations, we propose in this letter a generative adversarial ConvGRU (GA-ConvGRU) model. The model is composed of two adversarial learning systems, which are a ConvGRU-based generator and a convolution neural network-based discriminator. The two systems are trained by playing a minimax game. With the adversarial learning scheme, GA-ConvGRU can yield more realistic and more accurate extrapolation. Experiments on real data sets have been conducted and the results demonstrate that the proposed GA-ConvGRU significantly outperforms state-of-the-art extrapolation methods ConvGRU and optical flow. [ABSTRACT FROM AUTHOR]

Road information extraction based on aerial images is a critical task for many applications, and it has attracted considerable attention from researchers in the field of remote sensing. The problem is mainly composed of two subtasks, namely, road detection and centerline extraction. Most of the previous studies rely on multistage-based learning methods to solve the problem. However, these approaches may suffer from the well-known problem of propagation errors. In this paper, we propose a novel deep learning model, recurrent convolution neural network U-Net (RCNN-UNet), to tackle the aforementioned problem. Our proposed RCNN-UNet has three distinct advantages. First, the end-to-end deep learning scheme eliminates the propagation errors. Second, a carefully designed RCNN unit is leveraged to build our deep learning architecture, which can better exploit the spatial context and the rich low-level visual features. Thereby, it alleviates the detection problems caused by noises, occlusions, and complex backgrounds of roads. Third, as the tasks of road detection and centerline extraction are strongly correlated, a multitask learning scheme is designed so that two predictors can be simultaneously trained to improve both effectiveness and efficiency. Extensive experiments were carried out based on two publicly available benchmark data sets, and nine state-of-the-art baselines were used in a comparative evaluation. Our experimental results demonstrate the superiority of the proposed RCNN-UNet model for both the road detection and the centerline extraction tasks. [ABSTRACT FROM AUTHOR]

Finding customer groups from transaction data is very important for retail and e-commerce companies. Recently, a “Purchase Tree” data structure is proposed to compress the customer transaction data and a local PurTree spectral clustering method is proposed to cluster the customer transaction data. However, in the PurTree distance, the node weights for the children nodes of a parent node are set as equal and the differences between different nodes are not distinguished. In this paper, we propose a two-level subspace weighting spectral clustering (TSW) algorithm for customer transaction data. In the new method, a PurTree subspace metric is proposed to measure the dissimilarity between two customers represented by two purchase trees, in which a set of level weights are introduced to distinguish the importance of different tree levels and a set of sparse node weights are introduced to distinguish the importance of different tree nodes in a purchase tree. TSW learns an adaptive similarity matrix from the local distances in order to better uncover the cluster structure buried in the customer transaction data. Simultaneously, it learns a set of level weights and a set of sparse node weights in the PurTree subspace distance. An iterative optimization algorithm is proposed to optimize the proposed model. We also present an efficient method to compute a regularization parameter in TSW. TSW was compared with six clustering algorithms on ten benchmark data sets and the experimental results show the superiority of the new method. [ABSTRACT FROM AUTHOR]

Books, as a representative of lengthy documents, convey rich semantics. Traditional document modeling methods, such as bag-of-words models, have difficulty capturing such rich semantics when only considering term-frequency features. In order to explore term spatial distributions over a book, a tree-structured book representation is investigated in this paper. Moreover, an efficient learning framework, Tree2Vector, is introduced for mapping tree-structured book data into vectorial space. In particular, we present two types of locality reconstruction (LR) models: Euclidean-type and cosine-type, during the transformation process of tree structures into vectorial representations. The LR is used for modeling the reconstruction process, in which each parent node in a tree is supposed to be reconstructed by its child nodes. The prominent advantage of this Tree2Vector framework is that it solely utilizes the local information within a single book tree. In addition, extensive experimental results demonstrate that Tree2Vector is able to deliver comparable or better performance in comparison to methods that consider the information of all trees in a database globally. Experimental results also suggest that cosine-type LR consistently performs better than Euclidean-type LR in applications of book and author recommendations. [ABSTRACT FROM AUTHOR]

Deep learning has achieved great successes in conventional computer vision tasks. In this paper, we exploit deep learning techniques to address the hyperspectral image classification problem. In contrast to conventional computer vision tasks that only examine the spatial context, our proposed method can exploit both spatial context and spectral correlation to enhance hyperspectral image classification. In particular, we advocate four new deep learning models, namely, 2-D convolutional neural network (2-D-CNN), 3-D-CNN, recurrent 2-D CNN (R-2-D-CNN), and recurrent 3-D-CNN (R-3-D-CNN) for hyperspectral image classification. We conducted rigorous experiments based on six publicly available data sets. Through a comparative evaluation with other state-of-the-art methods, our experimental results confirm the superiority of the proposed deep learning models, especially the R-3-D-CNN and the R-2-D-CNN deep learning models. [ABSTRACT FROM AUTHOR]

Opportunistic mobile social network (OMSN) enables users to form an instant social network for information sharing (e.g., people watching the same soccer game can share their instant comments). OMSN is ad hoc in nature, thus relies on the cooperation of members regarding message transmission. However, some uncooperative or malicious behavior from abnormal members may reduce network performance, even damage the entire network. Currently, there does not exist effective mechanisms to detect selfish and malicious nodes. To tackle this problem, we propose a dynamic trust framework to facilitate a node to derive a trust value of another node based on the behavior of the latter. The novelty of our framework includes the following: 1) we design a new metric for a trust value of a node and 2) we propose a “two–hop feedback method” that requires intermediate nodes in a forwarding path to generate ACK messages to verify a node’s honesty if they are two hops away. In most existing trust models, final ACK messages are considered as critical factors. In OMSN, nodes are not fully connected and final ACK messages cannot be reliably received. In order to avoid the problem that few final ACK messages can be received, we propose a “two–hop feedback method.” Simulation results show that our approach is able to detect a majority of abnormal nodes including malicious nodes, selfish nodes, and those nodes launching conspiracy attacks. Thus, the entire network efficiency can be improved without negative impact of abnormal nodes. Besides, our trust framework can be easily applied to the current popular routing protocols of opportunistic networks. [ABSTRACT FROM PUBLISHER]

Learning object detection models from weakly labeled data is an important topic in computer vision. Among various types of weak annotations, image-level object labeling is a natural one that tells the existence, but not the precise locations, of object instances in images. Learning object detectors from image-level labels can be naturally cast as a multiple instance learning (MIL) problem. Existing MIL approaches for object detection still suffer from high false positive rates due to the lack of advanced instances selection techniques. In this study, a subspace-based generative model is proposed to select positive instances by minimizing rank of the coefficient matrix associated with the subspace models. An incoherence term between the subspace model and some “hard” negative instances in then modeled by an $\epsilon$-insensitive loss function. To further improve the discriminative ability, an ensemble strategy is proposed by employing multiple subspace models. Rigorous experiments are performed on several datasets, and the promising experimental results demonstrate that the proposed approach is superior to the state-of-the-art weakly supervised learning algorithms in terms of precision, recall, and F-score. [ABSTRACT FROM PUBLISHER]

A new triple wing harmonium (TWH) model that integrates text metadata into a low-dimensional semantic space is proposed for the application of content-based movie recommendation. The text metadata considered here include movie synopsis, actor list, and user comments. We develop a new TWH model projecting these multiple textual features into low-dimensional latent topics with different probability distribution assumptions. A contrastive divergence (CD) algorithm is used for efficient learning and inference. Experimental results suggest that the proposed method performs better than the state-of-the-art algorithms for movie recommendation. [ABSTRACT FROM PUBLISHER]