Search

The deep learning based methods have improved the visual tracking precision significantly. However, the background distraction and the high precise localization remain challenging problems. Despite that some methods have fused the deep and shallow layer features to solve these problems, the existing fusion methods, like simply concatenating or adding the features from the different layers, cannot take the advantage of both the deep and shallow layer features fully. In this paper, we propose a new adaptive feature fusion method, called the instance-based feature pyramid (IBFP) to obtain the discriminative high-resolution feature, which not only inherits the discriminative information from the deep layer feature, but also keeps the high precision localization information of the shallow layer feature. For utilizing the deep and shallow features effectively, we design an instance-based upsampling (IBU) module to fuse them, and a compressed space channel selection (CSCS) module to re-weight the feature channels adaptively. We insert the IBU and CSCS modules in the Siamese tracker for end-to-end training and testing. By using the proposed IBU and CSCS modules, we fuse the deep and shallow features in a series manner. Experiments on large-scale benchmark datasets demonstrate that the proposed modules boost the capabilities of distinguishing the targets and the similar distractors and perform favorably against the state-of-the-art. [ABSTRACT FROM AUTHOR]

Margin-based metric losses have shown great success in Person Re-identification and Face Verification. But most existing works adopt a fixed class-level margin regardless of the difference between each training sample. This paper proposes a Norm-Aware Margin Assignment (NAMA) scheme to dynamically adjust the weight of each sample during training. Combined with the existing margin-based classification losses, NAMA improves the robustness of feature embedding by assigning larger margins to more recognizable samples. NAMA is a fully trainable module that automatically models the correlation between the optimal margin and image quality during back-propagation without supervision. To stabilize the training and make the assigned margin more controllable, we introduce a margin re-balance mechanism to align the expectation of learned margins to a pre-defined value. Extensive experiments on three popular ReID benchmarks validate the effectiveness of our NAMA method. Code will be publicly available at: https://github.com/huangzongheng/NAMA. [ABSTRACT FROM AUTHOR]

In this article, we propose a novel compressed latent distribution representation for 3D hand pose estimation from monocular RGB images to alleviate the channel correspondence problem. The channel correspondence problem occurs when the 2D and depth coordinates are estimated from independent feature maps, which means the 2D and depth channel sequences may not match during the cross-dataset inference. In contrast, we propose a compressed latent distribution representation that the 2D and depth feature maps for each joint are interconnected and inter-constrained more directly, effectively alleviating the channel correspondence problem and improving cross-dataset performance. Moreover, we design an efficient encoder-decoder network that can maintain the resolution of feature maps to enable better hand feature extraction from monocular RGB images. In this work, the overall pipeline contains two branches: one is the 2D hand pose estimation branch based on a latent heatmap representation (LHR); the other is the 3D hand pose estimation branch based on our proposed latent distribution representation (LDR). In this way, the 2D estimation branch serves as guidance for the 3D branch, which simplifies the optimization of the overall network and results in a more rapid convergence during training. The results on several benchmark datasets (including STB, RHD, and the most recently released InterHand2.6M) demonstrate that our proposed method achieves state-of-the-art (SOTA) performance. [ABSTRACT FROM AUTHOR]

Fine-grained information has been proved helpful for person re-identification, and multi-head attention mechanism offers a feasible solution for this. However, we observe severe redundancy among the multiple branches, which might make the learned representation over-emphasize certain discriminative regions and correspondingly ignore other potentially informative regions. Therefore, we tackle this issue by two aspects yielding the so-called Complementation-Reinforced Attention Network (CRAN). One is the redundancy among branches, and we propose to impose complementing constraints among multiple attention heads. The constraints are two-fold: on the one hand, it encourages each branch to attend to complementary attention regions; on the other hand, it enforces orthogonality among the learned features of different regions in the embedding space. The other is the redundancy among query positions for each attention head. So we simplify the attention block by sparsifying the query positions. Besides, in order to achieve efficient retrieval, we propose an adaptive feature fusion method for dimensional reduction. Compared with the commonly used feature ensemble, our method effectively reduces the dimensionality while keeping the discriminative ability. We demonstrate the effectiveness of our method on MSMT17, Market-1501, DukeMTMC-reID, and CUHK03 datasets. [ABSTRACT FROM AUTHOR]

In this paper, we address the challenging problem of weakly supervised spatio-temporal action localization for which only video-level action labels are available during training. To solve this problem, we propose an end-to-end Global Local Network (GLNet) to predict the probability distribution simultaneously in both spatial and temporal space. The proposed GLNet model includes two key components: a local spatial module and a global temporal module. The local spatial module aims to predict the frame-level spatial distribution by encoding short-term temporal information. In particular, we propose a Region Actionness Network (RAN) to select the target region boxes from the precomputed exhaustive proposals. The global temporal module can predict temporal distribution by a long-term temporal structure modelling. Specifically, we design a temporal fusion-and-excitation architecture on the top of several clips, and trained by a sparse loss function. Therefore, the proposed GLNet model can perform spatio-temporal action localization in an end-to-end manner. We evaluate the performance of GLNet on the J-HMDB and UCF101-24 datasets. The experimental results demonstrate GLNet achieves a significant margin against other state-of-the-art weakly supervised methods and even some fully supervised methods in terms of frame mean Average Precision (mAP) and the video mAP (called frame-mAP and video-mAP, respectively). [ABSTRACT FROM AUTHOR]

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks. [ABSTRACT FROM AUTHOR]

Semantic parts have shown a powerful discriminative capacity for action recognition. However, many existing methods select parts according to predefined heuristic rules, which may cause the correlation among parts to be lost, or do not appropriately consider the cluttered candidate part space, which may result in weak generalizability of the resulting action labels. Therefore, better consideration of the correlation among parts and refinement of the candidate space will lead to a more discriminative action representation. This paper achieves improved performance by more elegantly addressing these two factors. First, considering the cluttered nature of the candidate space, we propose a recursive part elimination strategy for iterative refinement of the candidate parts. In each iteration, we eliminate the parts with the lowest weights, which are deemed to be noise. Second, we measure the discriminative capabilities of the candidates and select the top-ranked parts by applying a maximum margin model, which can alleviate overfitting while simultaneously improving generalizability and correlation extraction. Finally, using the selected parts, we extract mid-level features. We report experiments conducted on four datasets (KTH, Olympic Sports, UCF50, and HMDB51). The proposed method can achieve significant improvements compared with other recent methods, including a lower computational cost, a faster speed, and higher accuracy. [ABSTRACT FROM PUBLISHER]

It is important to track people's face efficiently and accurately in many Intelligent Transportation Systems (ITSs) and Safety Driving Assistant Systems (SDASs). This paper presents a high-performance and low-cost real-time face tracking system, which runs on general onboard computer with very low CPU consumption. The proposed face tracking system is composed of four modules: the motion detector, face detector, face tracker, and face validator. The motion detector extracts motion areas by using a spatial-temporal bi-differential method with a very low computational cost. The face detector integrates motions into a cascade face detection framework to reject most of non-face scanning-windows to ensure efficient face localization. The face tracker fuses motion feature with color feature to alleviate the drifting problem during tracking. The face validator builds face appearance models online and identifies each specific tracked face to avoid confusion. Experimental results on three challenging video sequences show that the proposed face tracking system outperforms the state-of-the-art face tracker and consumes only 5-13% CPU resources of a low-spec onboard computer while processing in real time. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Mid-level parts are shown to be effective for human action recognition in videos. Typically, these semantic parts are first mined with some heuristic rules, then videos are represented via volumetric max-pooling (VMP) method. However, these methods have two issues: 1) the VMP strategy divides videos by static grids. In this case, a semantic part may occur in different localizations in different videos. That means the VMP strategy loses the space-time invariance. To solve this problem, we propose to apply a saliency-driven max-pooling scheme to represent a video. We extract the video semantic cues by the saliency map, and dynamically pool the local maximum responses. This scheme can be considered as a semantic content-based feature alignment method and 2) the parts discovered by heuristic rules may be intuitive but not discriminative enough for action classification because they neglect the relations between the detectors. For this issue, we propose to apply a sparse classifier model to select discriminative parts. Moreover, to further improve the discriminative ability of the representation, we propose to conduct feature selection by the corresponding entry magnitude of the model coefficients. We conduct experiments on four challenging datasets—KTH, Olympic Sports, UCF50, and HMDB51. The results show that the proposed method significantly outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Person reidentification (re-id) aims to match a specific person across nonoverlapping cameras, which is an important but challenging task in video surveillance. Conventional methods mainly focus either on feature constructing or metric learning. Recently, some deep learning-based methods have been proposed to learn image features and similarity measures jointly. However, current deep models for person re-id are usually trained with either pairwise loss, where the number of negative pairs greatly outnumbering that of positive pairs may lead the training model to be biased toward negative pairs or constant margin hinge loss, without considering the fact that hard negative samples should be paid more attention in the training stage. In this paper, we propose to learn deep representations with an adaptive margin listwise loss. First, ranking lists instead of image pairs are used as training samples, in this way, the problem of data imbalance is relaxed. Second, by introducing an adaptive margin parameter in the listwise loss function, it can assign larger margins to harder negative samples, which can be interpreted as an implementation of the automatic hard negative mining strategy. To gain robustness against changes in poses and part occlusions, our architecture combines four convolutional neural networks, each of which embeds images from different scales or different body parts. The final combined model performs much better than each single model. The experimental results show that our approach achieves very promising results on the challenging CUHK03, CUHK01, and VIPeR data sets. [ABSTRACT FROM PUBLISHER]

This letter presents a novel multi-target tracking approach consisting of two parts. The first part is the detection based association to form global tracks. Short yet reliable tracklets are firstly generated. By effectively combining appearance and motion information, a Hough forest learning framework is constructed to obtain a more discriminative affinity model and produce longer association between tracklets. In the second part, in order to connect isolate detections for trajectory consistency, we present an appearance similarity model based on mutual occlusion reasoning. A novel fusion feature template is designed to accurately compute the matching score between each isolated detection and target. Experimental results show significant improvements of our method when compared with several state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Covariance tracking has achieved impressive successes in recent years due to its competent region covariance-based feature descriptor. Although adopt fast integral image computation, covariance tracking's brute-force search strategy is still inefficient and it possibly leads to inconsecutive tracking trajectory and distraction. In this work, a generalized, adaptive covariance tracking approach with novel integral region computation and occlusion detection is proposed. The integral region is much faster than integral image and adaptive to the tracking target and tracking condition. The adaptive search window can be adjusted dynamically by simple occlusion detection. The integral image and the global covariance tracking can be seen just a special case of integral region and the proposed approach, respectively. The proposed approach unifies the local and global search strategies in an elegant way and smoothly switches between them according to the tracking conditions (i.e. occlusion distraction or sudden motion) which are judged by occlusion detector. It gets much better efficiency, robustness of distraction, and stable trajectory by local search in normal steady state, while obtains more abilities for occlusion and re-identification by enlarged search window (until to global search) in abnormal situation at the same time. Our approach shows excellent target representation ability, faster speed, and more robustness, which has been verified on some video sequences. [ABSTRACT FROM AUTHOR]

Few-shot classification (FSC) is a highly challenging task, as only a small number of labeled samples are available when identifying new categories. Distance metric learning-based methods have emerged as a prominent approach to FSC, which typically use a distance function to measure the difference between query and support samples for identifying the class membership of the query sample. However, these methods simply treat each channel difference between query and support features equally when computing the class scores. Since different channels in the learned feature seek for different patterns, these distance metrics fail to consider that different channels are of different importance to FSC, and thus cannot accurately measure the similarity between samples. To address this issue, we propose a Q uery- C entric D istance M odulator (QCDM) to generate query-related weights for each channel difference adaptively. Specifically, since the distribution of difference between a query sample and all support samples in a particular channel can reflect the importance of this channel to the classification of the query sample, we take this difference vector as input and generate a query-specific channel weight through a meta-network. QCDM can guide FSC models to focus on discriminative channel differences and achieve better generalization. QCDM is a plug-and-play module that can be seamlessly integrated with existing distance metric learning-based FSC methods. Extensive experimental results indicate that our method can effectively improve the performance of distance metric learning-based FSC methods. The source code is available in https://github.com/Wu-Wenxiao/QCDM. • A plug-and-play query-centric distance modulator for distance metric learning-based FSC methods. • Weight generation through data inconsistency within each channel from different classes. • Decoupled from the process of feature extraction and integrated with any given backbone. [ABSTRACT FROM AUTHOR]

Person re-identification (re-id), aiming to search a specific person among a non-overlapping camera network, has attracted plenty of interest in recent years. This task is highly challenging, especially when there exists only single image per person in the database. In this paper, we present an algorithm for learning a Mahalanobis distance for person re-identification. Our method has two distinctive features: (1) to obtain the best separability of the training data, we first minimize the intra-class distances to the most extent by forcing intra-class distances to be zero, and (2) to promote the generalization ability of the learned metric, we then maximize the minimum margin between different classes. Inspired by the simple geometric intuition that a regular simplex maximizes its minimum side length, provided the sum of all side length is fixed, our method, called EquiDistance constrained Metric Learning (EquiDML), applies least-square regression technique to map images of the same person to the same vertex of a regular simplex, and images of different persons to different vertices of a regular simplex. Consequently, under the learned metric, images of the same class are collapsed to a single point, while images of different classes are transformed to be equidistant. This simple motivation is further formulated as a convex optimization problem, solved by the projected gradient descent method and proved to be very effective in person re-identification task. Although it is fairly simple, our method outperforms the state-of-the-art methods on CUHK01, CUHK03, Market1501 and DukeMTMC-reID datasets, and achieves very competitive performance on the widely used VIPeR dataset. [ABSTRACT FROM AUTHOR]