Search

Heterogeneous domain adaptation (HDA) tackles the learning of cross-domain samples with both different probability distributions and feature representations. Most of the existing HDA studies focus on the single-source scenario. In reality, however, it is not uncommon to obtain samples from multiple heterogeneous domains. In this article, we study the multisource HDA problem and propose a conditional weighting adversarial network (CWAN) to address it. The proposed CWAN adversarially learns a feature transformer, a label classifier, and a domain discriminator. To quantify the importance of different source domains, CWAN introduces a sophisticated conditional weighting scheme to calculate the weights of the source domains according to the conditional distribution divergence between the source and target domains. Different from existing weighting schemes, the proposed conditional weighting scheme not only weights the source domains but also implicitly aligns the conditional distributions during the optimization process. Experimental results clearly demonstrate that the proposed CWAN performs much better than several state-of-the-art methods on four real-world datasets.

The near-surface temperature prediction (NTP) is an important spatial-temporal forecast problem, which can be used to prevent temperature crises. Most of the previous approaches fail to explicitly model the long- and short-range spatial correlations simultaneously, which is critical to making an accurate temperature prediction. In this study, both long- and short-range spatial correlations are captured to fill this gap by a novel convolution operator named Long- and Short-range Convolution (LS-Conv). The proposed LS-Conv operator includes three key components, namely, Node-based Spatial Attention (NSA), Long-range Adaptive Graph Constructor (LAGC), and Long- and Short-range Integrator (LSI). To capture long-range spatial correlations, NSA and LAGC are proposed to evaluate node importance aiming at auto-constructing long-range spatial correlations, which is named as Long-range aware Graph Convolution Network (LR-GCN). After that, the Short-range aware Convolution Neural Network (SR-CNN) accounts for the short-range spatial correlations. Finally, LSI is proposed to capture both long- and short-range spatial correlations by intra-unifying LR-GCN and SR-CNN. Upon the proposed LS-Conv operator, a new model called Long- and Short-range for NPT (LS-NTP) is developed. Extensive experiments are conducted on two real-world datasets and the results demonstrate that the proposed method outperforms state-of-the-art techniques. The source code is available on GitHub:https://github.com/xuguangning1218/LS_NTP.

A typhoon is a destructive weather system that can cause severe casualties and economic losses. Typhoon intensity (TI) is a measurement to evaluate its ruinous degree. Hence, typhoon intensity prediction is an important research problem and many methods have been proposed. However, most of the existing approaches have very limited capability to combine the 2D Typhoon Structure Domain-expert Knowledge (2D-TSDK) and the 3D Typhoon Structure Data-driven Knowledge (3D-TSDK) for the TI prediction. To address this issue, this paper proposes a spatio-temporal deep learning method named Spatial Attention Fusing Network (SAF-Net). The designed model aims to fuse the 2D-TSDK and the 3D-TSDK by developing a specific Wide & Deep framework. In the data-driven component, a special Spatial Attention (SA) module is designed to automatically select high-response wind speed areas and embedded into a three-branch CNN to exploit the 3D-TSDK. Then, the Wide & Deep framework integrates the 2D-TSDK and the 3D-TSDK for the TI prediction. Comprehensive experiments have been conducted on a real-world dataset, and the result shows that the proposed method outperforms state-of-the-art typhoon intensity prediction methods. The code is available in GitHub: https://github.com/xuguangning1218/TI_Prediction

Due to the degeneration problem of sensors, calibration becomes a prerequisite step to retrieve consistent satellite images, especially for the ones from long-term time series. Relative calibration is an economic manner to address the problem. Previous studies leverage the identified no-change pixels (NCPs) between two images for relative calibration. However, the identification of NCPs itself is a very hard task and the inferior detection quality affects the performances significantly. Inspired by the great success of deep learning techniques, in this article, we first develop a convolutional neural network (CNN)-based relative calibration method, which bypasses the NCP detection. In particular, the ratio of sensor sensitivity coefficients at two time points is directly estimated by feeding the corresponding image pair into our developed CNN regressor. A polynomial function is fitted upon the estimated ratios in time series. We train the CNN regressor based on the multisite calibration results and then conduct experiments on FengYun-3A (FY-3A), FengYun-3B (FY-3B), and FengYun-3C (FY-3C). The results validate the effectiveness of the proposed method, and it outperforms state-of-the-art NCP-based methods.

Poverty prediction is an important task for developing countries that lack the key measures of economic development. The prediction can help governments to allocate scarce resources for sustainable development. Nighttime satellite imagery offers an opportunity to address the task. However, as the nighttime satellite data contain a large amount of noise, directly leveraging it is not very effective. Previous studies have shown that relying on deep learning techniques nighttime satellite data can be a good proxy between daytime satellite imagery and the poverty index. In this letter, based on the proxy, we leverage four deep learning approaches, namely, VGG-Net, Inception-Net, ResNet, and DenseNet, to extract deep features from daytime satellite imagery and then apply least absolute shrinkage and selection operator (LASSO) regression for poverty prediction. To further enhance the performance, we also integrate the squeeze and excitation (SE) module and focal loss into ResNet and DenseNet. Experimental results demonstrate the effectiveness of the investigated approaches, and the DenseNet with SE module and focal loss performs the best.

Few-shot learning (FSL) aims to recognize a novel class with very few instances, which is a challenging task since it suffers from a data scarcity issue. One way to effectively alleviate this issue is introducing explicit knowledge summarized from human past experiences to achieve knowledge transfer for FSL. Based on this idea, in this paper, we introduce the explicit knowledge of class hierarchy (i.e., the hierarchy relations between classes) as FSL priors and propose a novel hyperbolic knowledge transfer framework for FSL, namely, HyperKT. Our insight is, in the hyperbolic space, the hierarchy relation between classes can be well preserved by resorting to the exponential growth characters of hyperbolic volume, so that better knowledge transfer can be achieved for FSL. Specifically, we first regard the class hierarchy as a tree-like structure. Then, 1) a hyperbolic representation learning module and a hyperbolic prototype inference module are employed to encode/infer each image and class prototype to the hyperbolic space, respectively; and 2) a novel hierarchical classification and relation reconstruction loss are carefully designed to learn the class hierarchy. Finally, the novel class prediction is performed in a nearest-prototype manner. Extensive experiments on three datasets show our method achieves superior performance over state-of-the-art methods, especially on 1-shot tasks.

Precipitation nowcasting is an important task, which can serve numerous applications such as urban alert and transportation. Previous studies leverage convolutional recurrent neural networks (RNNs) to address the problem. However, they all suffer from two inherent drawbacks of the convolutional RNN, namely, the lack of a memory cell to preserve the fine-grained spatial appearances and the position misalignment issue when combining current observations with previous hidden states. In this article, we aim to overcome the defects. Specifically, we propose a novel pseudo flow spatiotemporal LSTM unit (PFST-LSTM), where a spatial memory cell and a position alignment module are developed and embedded in the structure of LSTM. Upon the PFST-LSTM units, we develop a new sequence-to-sequence architecture for precipitation nowcasting, which can effectively combine the spatial appearances and motion information. Extensive empirical evaluations are conducted on synthetic MovingMNIST++ and CIKM AnalytiCup 2017 datasets. Our experimental results demonstrate the superiority of the proposed PFST-LSTM over the state-of-the-art competitors. To reproduce the results, we release the source code at: https://github.com/luochuyao/PFST-LSTM .

Reinforcement learning (RL) techniques have recently been introduced to recommender systems. Most existing research works focus on designing policy and learning algorithms of the recommender agent but seldom care about the top-aware issue, i.e., the performance on the top positions is not satisfying, which is crucial for real applications. To address the drawback, we propose a Supervised deep Reinforcement learning Recommendation framework named as SRR. Within this framework, we utilize a supervised learning (SL) model to partially guide the learning of recommendation policy, where the supervision signal and RL signal are jointly employed and updated in a complementary fashion. We empirically find that suitable weak supervision helps to balance the immediate reward and the long-term reward, which nicely addresses the top-aware issue in RL based recommendation. Moreover, we perform a further investigation on how different supervision signals impact on recommendation policy. Extensive experiments are carried out on two real-world datasets under both the offline and simulated online evaluation settings, and the results demonstrate that the proposed methods indeed resolve the top-aware issue without much performance sacrifice in the long-run, compared with the state-of-the-art methods.

Precipitation nowcasting is an important task in operational weather forecasts. The key challenge of the task is the radar echo map extrapolation. The problem is mainly solved by an optical-flow method in existing systems. However, the method cannot model rapid and nonlinear movements. Recently, a convolutional gated recurrent unit (ConvGRU) method is developed, which aims to model such movements based on deep learning techniques. Despite the promising performance, ConvGRU tends to yield blurring extrapolation images and fails to multi-modal and skewed intensity distribution. To overcome the limitations, we propose in this letter a generative adversarial ConvGRU (GA-ConvGRU) model. The model is composed of two adversarial learning systems, which are a ConvGRU-based generator and a convolution neural network-based discriminator. The two systems are trained by playing a minimax game. With the adversarial learning scheme, GA-ConvGRU can yield more realistic and more accurate extrapolation. Experiments on real data sets have been conducted and the results demonstrate that the proposed GA-ConvGRU significantly outperforms state-of-the-art extrapolation methods ConvGRU and optical flow.

Aspect-based (aspect-level) sentiment analysis is an important task in fine-grained sentiment analysis, which aims to automatically infer the sentiment towards an aspect in its context. Previous studies have shown that utilizing the attention-based method can effectively improve the accuracy of the aspect-based sentiment analysis. Despite the outstanding progress, aspect-based sentiment analysis in the real-world remains several challenges. (1) The current attention-based method may cause a given aspect to incorrectly focus on syntactically unrelated words. (2) Conventional methods fail to identify the sentiment with the special sentence structure, such as double negatives. (3) Most of the studies leverage only one vector to represent context and target. However, utilizing one vector to represent the sentence is limited, as the natural languages are delicate and complex. In this paper, we propose a knowledge guided capsule network (KGCapsAN), which can address the above deficiencies. Our method is composed of two parts, a Bi-LSTM network and a capsule attention network. The capsule attention network implements the routing method by attention mechanism. Moreover, we utilize two prior knowledge to guide the capsule attention process, which are syntactical and n-gram structures. Extensive experiments are conducted on six datasets, and the results show that the proposed method yields the state-of-the-art.

Cross-modal hashing is an important technology for large-scale very high resolution (VHR) and synthetic-aperture radar (SAR) image retrieval. Current cross-modal hashing methods fail to effectively preserve the intra-class similarities and the inter-class discriminations between VHR and SAR images when learning common semantic representation of these cross-modal images. This is because these methods use derived signals to implicitly guide hashing learning, which leads to low discrimination of generated hash codes. To address the drawback, this paper proposes an explicit semantic preserving-based deep hashing method, which can fully learn the intra-class and inter-class semantic structure. Specifically, we design a novel objective function to explicitly preserve the intra-class and inter-class semantic structure directly with class labels. Extensive experiments on a VHR-SAR dataset demonstrate that our method outperforms various state-of-the-art cross-modal hashing methods.

Wind power is a new type of green energy. Though it is economical to access and gather such energy, effectively matching the energy with consumers' demand is difficult, because of the fluctuate, intermittent and chaotic nature of wind speed. Hence, multi-step wind speed prediction becomes an important research topic. In this paper, we propose a novel deep learning method, DyanmicNet, for the problem. DynamicNet follows an encoder-decoder framework. To capture the fluctuate, intermittent and chaotic nature of wind speed, it leverages a time-variant structure to build the decoder, which is different from conventional encoder-decoder methods. In addition, a new neural block (ST-GRU-ODE) is developed, which can model the wind speed in a continuous manner by using the neural ordinary differential equation (ODE). To enhance the prediction performance, a multi-step training procedure is also put forward. Comprehensive experiments have been conducted on two real-world datasets, where wind speed is recorded in the form of two orthogonal components namely U-Wind and V-Wind. Each component can be illustrated as wind speed images. Experimental results demonstrate the effectiveness and superiority of the proposed method over state-of-the-art techniques.

Finding customer groups from transaction data is very important for retail and e-commerce companies. Recently, a "Purchase Tree" data structure is proposed to compress the customer transaction data and a local PurTree spectral clustering method is proposed to cluster the customer transaction data. However, in the PurTree distance, the node weights for the children nodes of a parent node are set as equal and the differences between different nodes are not distinguished. In this paper, we propose a two-level subspace weighting spectral clustering (TSW) algorithm for customer transaction data. In the new method, a PurTree subspace metric is proposed to measure the dissimilarity between two customers represented by two purchase trees, in which a set of level weights are introduced to distinguish the importance of different tree levels and a set of sparse node weights are introduced to distinguish the importance of different tree nodes in a purchase tree. TSW learns an adaptive similarity matrix from the local distances in order to better uncover the cluster structure buried in the customer transaction data. Simultaneously, it learns a set of level weights and a set of sparse node weights in the PurTree subspace distance. An iterative optimization algorithm is proposed to optimize the proposed model. We also present an efficient method to compute a regularization parameter in TSW. TSW was compared with six clustering algorithms on ten benchmark data sets and the experimental results show the superiority of the new method.

Road information extraction based on aerial images is a critical task for many applications, and it has attracted considerable attention from researchers in the field of remote sensing. The problem is mainly composed of two subtasks, namely, road detection and centerline extraction. Most of the previous studies rely on multistage-based learning methods to solve the problem. However, these approaches may suffer from the well-known problem of propagation errors. In this paper, we propose a novel deep learning model, recurrent convolution neural network U-Net (RCNN-UNet), to tackle the aforementioned problem. Our proposed RCNN-UNet has three distinct advantages. First, the end-to-end deep learning scheme eliminates the propagation errors. Second, a carefully designed RCNN unit is leveraged to build our deep learning architecture, which can better exploit the spatial context and the rich low-level visual features. Thereby, it alleviates the detection problems caused by noises, occlusions, and complex backgrounds of roads. Third, as the tasks of road detection and centerline extraction are strongly correlated, a multitask learning scheme is designed so that two predictors can be simultaneously trained to improve both effectiveness and efficiency. Extensive experiments were carried out based on two publicly available benchmark data sets, and nine state-of-the-art baselines were used in a comparative evaluation. Our experimental results demonstrate the superiority of the proposed RCNN-UNet model for both the road detection and the centerline extraction tasks.

Most of existing image categorizations assume that the given datasets have a good resolution and quality. However, the assumption is often violated in real applications. In this paper, we study the low-resolution (LR) image categorization. By utilizing labeled high-resolution (HR) images as auxiliary information, we formulate the problem as a heterogeneous domain adaptation problem and propose a Discriminative Joint Distribution Adaptation (DJDA) model to solve it. The DJDA model projects both LR and HR images into an intermediate subspace with a well-designed objective function, where the distance between classes is expected to be enlarged and the distribution divergence to be reduced. As a result, the discriminative knowledge for HR images can be transferred effectively to LR images. Experimental results demonstrate the proposed DJDA method produces significantly superior categorization accuracies against state-of-the-art competitors.

Network embedding aims to learn low-dimensional representations for nodes in social networks, which can serve many applications, such as node classification, link prediction and visualization. Most of network embedding methods focus on learning the representations solely from the topological structure. Recently, attributed network embedding, which utilizes both the topological structure and node content to jointly learn latent representations, becomes a hot topic. However, previous studies obtain the joint representations by directly concatenating the one from each aspect, which may lose the correlations between the topological structure and node content. In this paper, we propose a new attributed network embedding method, TLVANE, which can address the drawback by exploiting the deep variational autoencoders (VAEs). Particularly, a two-level VAE model is built, where the first-level accounts for the joint representations while the second for the embeddings of each aspect. Extensive experiments on three real-world datasets have been conducted, and the results demonstrate the superiority of the proposed method against state-of-the-art competitors.

Books, as a representative of lengthy documents, convey rich semantics. Traditional document modeling methods, such as bag-of-words models, have difficulty capturing such rich semantics when only considering term-frequency features. In order to explore term spatial distributions over a book, a tree-structured book representation is investigated in this paper. Moreover, an efficient learning framework, Tree2Vector, is introduced for mapping tree-structured book data into vectorial space. In particular, we present two types of locality reconstruction (LR) models: Euclidean-type and cosine-type, during the transformation process of tree structures into vectorial representations. The LR is used for modeling the reconstruction process, in which each parent node in a tree is supposed to be reconstructed by its child nodes. The prominent advantage of this Tree2Vector framework is that it solely utilizes the local information within a single book tree. In addition, extensive experimental results demonstrate that Tree2Vector is able to deliver comparable or better performance in comparison to methods that consider the information of all trees in a database globally. Experimental results also suggest that cosine-type LR consistently performs better than Euclidean-type LR in applications of book and author recommendations.

Wildfires are a serious disaster, which often cause severe damages to forests and plants. Without an early detection and suitable control action, a small wildfire could grow into a big and serious one. The problem is especially fatal at night, as firefighters in general miss the chance to detect the wildfires in the very first few hours. Low-light satellites, which take pictures at night, offer an opportunity to detect night fire timely. However, previous studies identify night fires based on threshold methods or conventional machine learning approaches, which are not robust and accurate enough. In this paper, we develop a new deep learning approach, which determines night fire locations by a pixel-level classification on low-light remote sensing image. Experimental results on VIIRS data demonstrate the superiority and effectiveness of the proposed method, which outperforms conventional threshold and machine learning approaches.

Few-Shot Remote Sensing Scene Classification (FSRSSC) is an important task, which aims to recognize novel scene classes with few examples. Recently, several studies attempt to address the FSRSSC problem by following few-shot natural image classification methods. These existing methods have made promising progress and achieved superior performance. However, they all overlook two unique characteristics of remote sensing images: (i) object co-occurrence that multiple objects tend to appear together in a scene image and (ii) object spatial correlation that these co-occurrence objects are distributed in the scene image following some spatial structure patterns. Such unique characteristics are very beneficial for FSRSSC, which can effectively alleviate the scarcity issue of labeled remote sensing images since they can provide more refined descriptions for each scene class. To fully exploit these characteristics, we propose a novel scene graph matching-based meta-learning framework for FSRSSC, called SGMNet. In this framework, a scene graph construction module is carefully designed to represent each test remote sensing image or each scene class as a scene graph, where the nodes reflect these co-occurrence objects meanwhile the edges capture the spatial correlations between these co-occurrence objects. Then, a scene graph matching module is further developed to evaluate the similarity score between each test remote sensing image and each scene class. Finally, based on the similarity scores, we perform the scene class prediction via a nearest neighbor classifier. We conduct extensive experiments on UCMerced LandUse, WHU19, AID, and NWPU-RESISC45 datasets. The experimental results show that our method obtains superior performance over the previous state-of-the-art methods., Comment: 13 pages

Few-shot learning aims to recognize novel classes with few examples. Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning. However, results show that the fine-tuning step makes marginal improvements. In this paper, 1) we figure out the reason, i.e., in the pre-trained feature space, the base classes already form compact clusters while novel classes spread as groups with large variances, which implies that fine-tuning feature extractor is less meaningful; 2) instead of fine-tuning feature extractor, we focus on estimating more representative prototypes. Consequently, we propose a novel prototype completion based meta-learning framework. This framework first introduces primitive knowledge (i.e., class-level part or attribute annotations) and extracts representative features for seen attributes as priors. Second, a part/attribute transfer network is designed to learn to infer the representative features for unseen attributes as supplementary priors. Finally, a prototype completion network is devised to learn to complete prototypes with these priors. Moreover, to avoid the prototype completion error, we further develop a Gaussian based prototype fusion strategy that fuses the mean-based and completed prototypes by exploiting the unlabeled samples. Extensive experiments show that our method: (i) obtains more accurate prototypes; (ii) achieves superior performance on both inductive and transductive FSL settings., Comment: Extended version of 'Prototype Completion with Primitive Knowledge for Few-Shot Learning' in CVPR2021. arXiv admin note: substantial text overlap with arXiv:2009.04960

The SEIR model is widely used in simulating the spread of infectious diseases. COVID-19 virus is a very severe infectious disease. Some studies leverage the SEIR or SEIRD model to simulate the spread and estimate the number of infected and recovered people as time goes on. However, these models suffer from two key deficiencies: (i) conventional SEIRD does not update its model parameters w.r.t. time; (ii) it focuses on predicting the trend, instead of the actual number of infections in the future. In this paper, we propose a cascade SEIRD model. The model learns and updates its parameters every day. Moreover, it is able to predict the number of infection cases, recovered cases and deaths. Specifically, we leverage a machine learning like approach to dynamically estimate the parameters of infection rate, incubation rate, recovery rate and death rate, which can be updated by gradient descent algorithm. Once the nature of the parameters w.r.t. time is determined, ARIMA model is adopted to characterize the dynamics of the parameters and predict their future changes. To validate the effectiveness of the proposed cascade SEIRD model, we conduct experiments on five data sets of different scales of regions (China, Hubei, Wuhan, Shenzhen, US). Experimental results show that the proposed cascade SEIRD achieves the most accurate prediction and outperforms state-of-the-art techniques.