Search

Multi-task learning (MTL) aims at tackling multiple tasks through one single network while guarantee all tasks can reach good performance. The main challenge in MTL is how to extract task-specific feature effectively. Existing task-specific feature extraction methods predominantly combine and stack convolutional neural networks (CNNs). However, these methods ignore two points: the geometric variations of the target object have different effects on each task; discriminative features for each task lack a mechanism to ensure they are focused on. In this work, we propose a Deformable-Attention Multi-Task Network (DAMTN) to improve the capability of extracting geometric invariance discriminative features. In particular, deformable convolution is introduced to learn geometric variation rules of the target object for different tasks, and attention mechanism helps task-specific networks focus on discriminative parts. The proposed DAMTN can be trained end-to-end. We empirically analyze the contribution of different components in the proposed method and demonstrate state-of-the-art performance on multiple classification tasks as well as semantic segmentation task and depth estimation task. [ABSTRACT FROM AUTHOR]

With the advance of Convolutional Neural Network, deep hashing methods have shown the great promising performance in large-scale image retrieval. Without depending on extensive human-annotated data, unsupervised hashing is more applicable to image retrieval tasks compared to supervised methods. However, due to the lack of fine-grained supervised signals and multi-similarity constraints, most state-of-the-art unsupervised deep hashing algorithms cannot ensure the correct fine-grained similarity ranking for image pairs. In this paper, we propose a novel unsupervised deep multi-similarity hashing framework to learn compact binary codes by jointly exploiting global-aware and spatial-aware representations, called Unsupervised Deep Multi-Similarity Hashing with Semantic Structure (UDMSH). Specifically, to obtain distinguishing characteristics, we develop a sub-network by jointly learning global semantic structures from Convolutional Neural Network (CNN) and inherent spatial structures from Fully Convolutional Network (FCN). By computing the cosine distance for deep features from image pairs, we construct a similarity matrix with semantic structure, then utilize this matrix to guide hash code learning process. Based on it, we carefully design a multi-level pairwise loss to preserve the correct fine-grained similarity ranking. Furthermore, we introduce Hamming-isometric mapping into unsupervised hashing framework to decrease the quantization errors. Extensive experiments on three widely used benchmarks prove that our proposed UDMSH outperforms several state-of-the-art unsupervised hashing with respect to different evaluation metrics. [ABSTRACT FROM AUTHOR]

Automatic image-based weather recognition has a great significance in the practical use of computer visual applications. The weather recognition approaches via multi-task learning obtain the description of weather conditions by jointly tackling weather-cue segmentation task and weather classification task. However, the existing approaches neglect the multiple scales of weather cues, which may decrease the ability to deal with both large and small weather cues in weather-cue segmentation. They also fail to assign appropriate weights, which the multi-task learning for weather recognition relies on, to the two tasks. To address these problems, we propose a novel end-to-end Convolutional Neural Network (CNN)-based Multi-task Weather Recognition Network with multi-scale weather cues (CMWRN). Specifically, in the weather-cue segmentation task, we propose to segment weather cues, e.g. blue sky and black clouds, by capturing multi-scale weather-cue features. Arbitrary-scale weather-cue regions in the outdoor images can be accurately and effectively classified via resampling the weather-cue features at multiple scales. In the weather classification task, a generalized pooling method is introduced to generate more discriminative weather representations. To further promote the weather classification, the multi-scale weather-cue features are exploited to provide weather clues for weather representations. Moreover, we develop an adaptive weighting scheme to automatically balance the weather classification task and weather-cue segmentation task for more effective training. Compared with the state-of-the-arts, experimental results on two public available benchmark datasets demonstrate the superior performance of our approach. • We capture multi-scale weather-cue features for multi-task weather recognition. • A generalized pooling is proposed to generate weather feature representations. • An adaptive weighting scheme is proposed for multi-task weather recognition. • The state-of-the-art results are reported on two weather recognition datasets. [ABSTRACT FROM AUTHOR]

With the explosive increase of remote sensing data, how to search for remote sensing data quickly and accurately in a vast dataset is an incredibly critical matter for research subjects. The deep hashing method has become the dominant method for remote sensing image retrieval because of its low-cost storage and high-speed retrieval. However, for the reason of the limitation of fixed convolutional kernels, deep hashing frameworks based on Convolutional Neural Networks (CNNs) fail to obtain the global semantic information well, which leads to the generation of suboptimal solutions. Furthermore, existing hashing methods commonly employ the random sampling strategy or hardest sample mining to build training batches, resulting in bad local minima. To remedy these problems, a novel Deep Global Semantic Structure-preserving Hashing framework via corrective triplet loss (DGSSH) is proposed for remote sensing image retrieval to learn a discriminative and stable embedding space, achieving intra-class confusion and inter-class diversity. Specifically speaking, the feature extraction module based on Swim Transformer architecture is developed to capture global semantic information and multiscale features from remote sensing images. Based on a distribution matching constraint, the corrective triplet loss for deep hashing schemes is designed to reduce the distribution shift caused by the random selection or hardest sample mining. Meanwhile, to reduce the time overhead of the model, the asymmetric learning strategy is employed to perform effective compact representation learning. Numerous experiments have been carried out on three publicly available benchmarks, which indicates that the proposed DGSSH framework could achieve optimal performance for remote sensing image retrieval applications. The source code of our DGSSH framework is hosted at https://github.com/QinLab-WFU/DGSSH.git. • The Swim Transformer architecture is developed to capture global semantic features. • The corrective triplet loss is proposed to reduce the distribution shift. • An asymmetric learning strategy is employed to improve training efficiency. • The state-of-the-art results are reported on three benchmark datasets. [ABSTRACT FROM AUTHOR]

Recently, many deep supervised hashing has been developed for multi-label image retrieval applications and has already achieved good effects. However, current methods quantify the similarities between image pairs by counting the number of shared labels roughly, totally ignoring the semantic relationships of image pairs. Besides, the sample imbalance of different classes could bias the training process toward majority categories, deteriorating the model performance. To solve the above-mentioned problems, we propose a novel deep supervised hashing framework, named Deep Multi-Similarity Hashing with semantic-aware preserving (DMSH), to generate binary codes with high-level semantic similarity. Specifically, to obtain distinguishing features, Convolutional Neural Networks (CNN) with correlation operators are introduced to capture discriminative semantic features of the input images, including auto-correlation characteristics and cross-correlation characteristics. By jointly exploiting label-level and semantic-level similarity, the semantic-aware similarity module is developed to quantify the high-level semantic similarity of image pairs, distinguishing the differences beyond categories. Furthermore, the multi-similarity loss is incorporated into deep hashing to collect informative pairs efficiently, which could alleviate the impact of imbalanced sample pairs during model training. Extensive comparison experiments on three benchmark datasets demonstrate that our DMSH achieves promising performance with respect to different evaluation metrics. • CNNs with correlation operators are introduced to capture higher-order features. • A semantic-aware method is proposed to quantify the similarity of image pairs. • Multi-similarity loss is developed to collect informative pairs efficiently. • The state-of-the-art results are reported on three benchmark datasets. [ABSTRACT FROM AUTHOR]