Search

The deep learning based methods have improved the visual tracking precision significantly. However, the background distraction and the high precise localization remain challenging problems. Despite that some methods have fused the deep and shallow layer features to solve these problems, the existing fusion methods, like simply concatenating or adding the features from the different layers, cannot take the advantage of both the deep and shallow layer features fully. In this paper, we propose a new adaptive feature fusion method, called the instance-based feature pyramid (IBFP) to obtain the discriminative high-resolution feature, which not only inherits the discriminative information from the deep layer feature, but also keeps the high precision localization information of the shallow layer feature. For utilizing the deep and shallow features effectively, we design an instance-based upsampling (IBU) module to fuse them, and a compressed space channel selection (CSCS) module to re-weight the feature channels adaptively. We insert the IBU and CSCS modules in the Siamese tracker for end-to-end training and testing. By using the proposed IBU and CSCS modules, we fuse the deep and shallow features in a series manner. Experiments on large-scale benchmark datasets demonstrate that the proposed modules boost the capabilities of distinguishing the targets and the similar distractors and perform favorably against the state-of-the-art. [ABSTRACT FROM AUTHOR]

Margin-based metric losses have shown great success in Person Re-identification and Face Verification. But most existing works adopt a fixed class-level margin regardless of the difference between each training sample. This paper proposes a Norm-Aware Margin Assignment (NAMA) scheme to dynamically adjust the weight of each sample during training. Combined with the existing margin-based classification losses, NAMA improves the robustness of feature embedding by assigning larger margins to more recognizable samples. NAMA is a fully trainable module that automatically models the correlation between the optimal margin and image quality during back-propagation without supervision. To stabilize the training and make the assigned margin more controllable, we introduce a margin re-balance mechanism to align the expectation of learned margins to a pre-defined value. Extensive experiments on three popular ReID benchmarks validate the effectiveness of our NAMA method. Code will be publicly available at: https://github.com/huangzongheng/NAMA. [ABSTRACT FROM AUTHOR]

In this article, we propose a novel compressed latent distribution representation for 3D hand pose estimation from monocular RGB images to alleviate the channel correspondence problem. The channel correspondence problem occurs when the 2D and depth coordinates are estimated from independent feature maps, which means the 2D and depth channel sequences may not match during the cross-dataset inference. In contrast, we propose a compressed latent distribution representation that the 2D and depth feature maps for each joint are interconnected and inter-constrained more directly, effectively alleviating the channel correspondence problem and improving cross-dataset performance. Moreover, we design an efficient encoder-decoder network that can maintain the resolution of feature maps to enable better hand feature extraction from monocular RGB images. In this work, the overall pipeline contains two branches: one is the 2D hand pose estimation branch based on a latent heatmap representation (LHR); the other is the 3D hand pose estimation branch based on our proposed latent distribution representation (LDR). In this way, the 2D estimation branch serves as guidance for the 3D branch, which simplifies the optimization of the overall network and results in a more rapid convergence during training. The results on several benchmark datasets (including STB, RHD, and the most recently released InterHand2.6M) demonstrate that our proposed method achieves state-of-the-art (SOTA) performance. [ABSTRACT FROM AUTHOR]

Fine-grained information has been proved helpful for person re-identification, and multi-head attention mechanism offers a feasible solution for this. However, we observe severe redundancy among the multiple branches, which might make the learned representation over-emphasize certain discriminative regions and correspondingly ignore other potentially informative regions. Therefore, we tackle this issue by two aspects yielding the so-called Complementation-Reinforced Attention Network (CRAN). One is the redundancy among branches, and we propose to impose complementing constraints among multiple attention heads. The constraints are two-fold: on the one hand, it encourages each branch to attend to complementary attention regions; on the other hand, it enforces orthogonality among the learned features of different regions in the embedding space. The other is the redundancy among query positions for each attention head. So we simplify the attention block by sparsifying the query positions. Besides, in order to achieve efficient retrieval, we propose an adaptive feature fusion method for dimensional reduction. Compared with the commonly used feature ensemble, our method effectively reduces the dimensionality while keeping the discriminative ability. We demonstrate the effectiveness of our method on MSMT17, Market-1501, DukeMTMC-reID, and CUHK03 datasets. [ABSTRACT FROM AUTHOR]

Stripe and random noise are two different degradation phenomena that commonly coexist in optical remote sensing images, and they are often modeled as inverse problems. In model-based inverse problems, analysis and synthesis sparse representations (SSRs) are used as regularization terms to obtain approximate solutions due to their respective merits, i.e., the nonzero coefficients in SSR are usually used to describe an image, while the indexes of zeros in analysis sparse representation (ASR) are used to characterize the stripe. Inspired by these merits, we propose a unified variational framework, called a joint analysis and weighted synthesis (JAWS) sparsity model, to simultaneously separate the clean image and the stripe from a single optical remote sensing image. To solve the JAWS sparsity model efficiently, an alternating minimization optimization strategy is first employed to separate it into two subproblems that are used for different tasks. One called as weighted SSR (WSSR) is the main for optical remote sensing image denoising, which can be effectively solved by employing the weighted singular value thresholding operator, while the other called as ASR is the main approach for optical remote sensing image destriping, which is optimized by adopting the split Bregman iteration. By minimizing the two subproblems alternatively, the proposed JAWS sparsity model is efficiently solved. Finally, both quantitative and qualitative results of experiments on synthetic and real-world optical remote sensing images validate that the proposed approach is effective and even better than the state of the arts. [ABSTRACT FROM AUTHOR]

In this paper, we address the challenging problem of weakly supervised spatio-temporal action localization for which only video-level action labels are available during training. To solve this problem, we propose an end-to-end Global Local Network (GLNet) to predict the probability distribution simultaneously in both spatial and temporal space. The proposed GLNet model includes two key components: a local spatial module and a global temporal module. The local spatial module aims to predict the frame-level spatial distribution by encoding short-term temporal information. In particular, we propose a Region Actionness Network (RAN) to select the target region boxes from the precomputed exhaustive proposals. The global temporal module can predict temporal distribution by a long-term temporal structure modelling. Specifically, we design a temporal fusion-and-excitation architecture on the top of several clips, and trained by a sparse loss function. Therefore, the proposed GLNet model can perform spatio-temporal action localization in an end-to-end manner. We evaluate the performance of GLNet on the J-HMDB and UCF101-24 datasets. The experimental results demonstrate GLNet achieves a significant margin against other state-of-the-art weakly supervised methods and even some fully supervised methods in terms of frame mean Average Precision (mAP) and the video mAP (called frame-mAP and video-mAP, respectively). [ABSTRACT FROM AUTHOR]

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks. [ABSTRACT FROM AUTHOR]

Dynamic scene blur is usually caused by object motion, depth variation as well as camera shake. Most existing methods usually solve this problem using image segmentation or fully end-to-end trainable deep convolutional neural networks by considering different object motions or camera shakes. However, these algorithms are less effective when there exist depth variations. In this work, we propose a deep neural convolutional network that exploits the depth map for dynamic scene deblurring. Given a blurred image, we first extract the depth map and adopt a depth refinement network to restore the edges and structure in the depth map. To effectively exploit the depth map, we adopt the spatial feature transform layer to extract depth features and fuse with the image features through scaling and shifting. Our image deblurring network thus learns to restore a clear image under the guidance of the depth map. With substantial experiments and analysis, we show that the depth information is crucial to the performance of the proposed model. Finally, extensive quantitative and qualitative evaluations demonstrate that the proposed model performs favorably against the state-of-the-art dynamic scene deblurring approaches as well as conventional depth-based deblurring algorithms. [ABSTRACT FROM AUTHOR]

We present an effective semi-supervised learning algorithm for single image dehazing. The proposed algorithm applies a deep Convolutional Neural Network (CNN) containing a supervised learning branch and an unsupervised learning branch. In the supervised branch, the deep neural network is constrained by the supervised loss functions, which are mean squared, perceptual, and adversarial losses. In the unsupervised branch, we exploit the properties of clean images via sparsity of dark channel and gradient priors to constrain the network. We train the proposed network on both the synthetic data and real-world images in an end-to-end manner. Our analysis shows that the proposed semi-supervised learning algorithm is not limited to synthetic training datasets and can be generalized well to real-world images. Extensive experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art single image dehazing algorithms on both benchmark datasets and real-world images. [ABSTRACT FROM AUTHOR]

Enhancement and denoising have always been a pair of conflicting problems in image processing of computer vision. Inspired by an earlier dual-domain filter (DDF), this letter proposes a progressive DDF to simultaneously enhance and denoise low-quality optical remote-sensing images. The main procedure of the proposed enhancement filter has two parts. First, a bilateral filter is exploited as a guide filter to obtain high-contrast images, which are enhanced by a histogram modification method. Then, low-contrast useful structures are restored by a short-time Fourier transform and are enhanced using an adaptive correction parameter. Both the quantitative and qualitative results of experiments on synthetic and real-world low-quality remote-sensing images demonstrate that the proposed method performs well on contrast enhancement, structure preservation, and noise reduction. Moreover, its satisfactory computation time resulting from its simple implementation makes it suitable for extensive application. [ABSTRACT FROM PUBLISHER]

Semantic parts have shown a powerful discriminative capacity for action recognition. However, many existing methods select parts according to predefined heuristic rules, which may cause the correlation among parts to be lost, or do not appropriately consider the cluttered candidate part space, which may result in weak generalizability of the resulting action labels. Therefore, better consideration of the correlation among parts and refinement of the candidate space will lead to a more discriminative action representation. This paper achieves improved performance by more elegantly addressing these two factors. First, considering the cluttered nature of the candidate space, we propose a recursive part elimination strategy for iterative refinement of the candidate parts. In each iteration, we eliminate the parts with the lowest weights, which are deemed to be noise. Second, we measure the discriminative capabilities of the candidates and select the top-ranked parts by applying a maximum margin model, which can alleviate overfitting while simultaneously improving generalizability and correlation extraction. Finally, using the selected parts, we extract mid-level features. We report experiments conducted on four datasets (KTH, Olympic Sports, UCF50, and HMDB51). The proposed method can achieve significant improvements compared with other recent methods, including a lower computational cost, a faster speed, and higher accuracy. [ABSTRACT FROM PUBLISHER]