Search

The deep learning based methods have improved the visual tracking precision significantly. However, the background distraction and the high precise localization remain challenging problems. Despite that some methods have fused the deep and shallow layer features to solve these problems, the existing fusion methods, like simply concatenating or adding the features from the different layers, cannot take the advantage of both the deep and shallow layer features fully. In this paper, we propose a new adaptive feature fusion method, called the instance-based feature pyramid (IBFP) to obtain the discriminative high-resolution feature, which not only inherits the discriminative information from the deep layer feature, but also keeps the high precision localization information of the shallow layer feature. For utilizing the deep and shallow features effectively, we design an instance-based upsampling (IBU) module to fuse them, and a compressed space channel selection (CSCS) module to re-weight the feature channels adaptively. We insert the IBU and CSCS modules in the Siamese tracker for end-to-end training and testing. By using the proposed IBU and CSCS modules, we fuse the deep and shallow features in a series manner. Experiments on large-scale benchmark datasets demonstrate that the proposed modules boost the capabilities of distinguishing the targets and the similar distractors and perform favorably against the state-of-the-art. [ABSTRACT FROM AUTHOR]

Margin-based metric losses have shown great success in Person Re-identification and Face Verification. But most existing works adopt a fixed class-level margin regardless of the difference between each training sample. This paper proposes a Norm-Aware Margin Assignment (NAMA) scheme to dynamically adjust the weight of each sample during training. Combined with the existing margin-based classification losses, NAMA improves the robustness of feature embedding by assigning larger margins to more recognizable samples. NAMA is a fully trainable module that automatically models the correlation between the optimal margin and image quality during back-propagation without supervision. To stabilize the training and make the assigned margin more controllable, we introduce a margin re-balance mechanism to align the expectation of learned margins to a pre-defined value. Extensive experiments on three popular ReID benchmarks validate the effectiveness of our NAMA method. Code will be publicly available at: https://github.com/huangzongheng/NAMA. [ABSTRACT FROM AUTHOR]

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks. [ABSTRACT FROM AUTHOR]

Dynamic scene blur is usually caused by object motion, depth variation as well as camera shake. Most existing methods usually solve this problem using image segmentation or fully end-to-end trainable deep convolutional neural networks by considering different object motions or camera shakes. However, these algorithms are less effective when there exist depth variations. In this work, we propose a deep neural convolutional network that exploits the depth map for dynamic scene deblurring. Given a blurred image, we first extract the depth map and adopt a depth refinement network to restore the edges and structure in the depth map. To effectively exploit the depth map, we adopt the spatial feature transform layer to extract depth features and fuse with the image features through scaling and shifting. Our image deblurring network thus learns to restore a clear image under the guidance of the depth map. With substantial experiments and analysis, we show that the depth information is crucial to the performance of the proposed model. Finally, extensive quantitative and qualitative evaluations demonstrate that the proposed model performs favorably against the state-of-the-art dynamic scene deblurring approaches as well as conventional depth-based deblurring algorithms. [ABSTRACT FROM AUTHOR]

We present an effective semi-supervised learning algorithm for single image dehazing. The proposed algorithm applies a deep Convolutional Neural Network (CNN) containing a supervised learning branch and an unsupervised learning branch. In the supervised branch, the deep neural network is constrained by the supervised loss functions, which are mean squared, perceptual, and adversarial losses. In the unsupervised branch, we exploit the properties of clean images via sparsity of dark channel and gradient priors to constrain the network. We train the proposed network on both the synthetic data and real-world images in an end-to-end manner. Our analysis shows that the proposed semi-supervised learning algorithm is not limited to synthetic training datasets and can be generalized well to real-world images. Extensive experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art single image dehazing algorithms on both benchmark datasets and real-world images. [ABSTRACT FROM AUTHOR]

Highlights • Motion blur kernel size is a significant input parameter of deblurring algorithms. • A regression CNN is proposed to estimate the kernel size of a blurry image. • The kernel size for training is labeled in a relative way. • The kernel size to be estimated can be arbitrary. • The estimated kernel size helps existing deblurring algorithms yielding satisfactory results. Abstract Deblurring is to restore a latent clear image as well as to estimate an underlying blur kernel from a single blurry image. Motion blur kernel size is a significant input parameter of existing deblurring algorithms. Setting the size manually, which is a tedious trial-and-error process, can hardly obtain a satisfactory result directly and effectively. In this paper, we formulate the kernel size estimation as a regression problem and construct a convolutional neural network (CNN) to resolve it. After training the CNN by a large number of simulated blurry images with labels, which are produced by a relative way, we can estimate the kernel sizes of a given blurry image without size limitation. Both quantitative and qualitative results of experiments show that our method can not only estimate the motion blur kernel size accurately but also help to yield better pictures with estimated kernel sizes as input parameters. [ABSTRACT FROM AUTHOR]

Person reidentification (re-id) aims to match a specific person across nonoverlapping cameras, which is an important but challenging task in video surveillance. Conventional methods mainly focus either on feature constructing or metric learning. Recently, some deep learning-based methods have been proposed to learn image features and similarity measures jointly. However, current deep models for person re-id are usually trained with either pairwise loss, where the number of negative pairs greatly outnumbering that of positive pairs may lead the training model to be biased toward negative pairs or constant margin hinge loss, without considering the fact that hard negative samples should be paid more attention in the training stage. In this paper, we propose to learn deep representations with an adaptive margin listwise loss. First, ranking lists instead of image pairs are used as training samples, in this way, the problem of data imbalance is relaxed. Second, by introducing an adaptive margin parameter in the listwise loss function, it can assign larger margins to harder negative samples, which can be interpreted as an implementation of the automatic hard negative mining strategy. To gain robustness against changes in poses and part occlusions, our architecture combines four convolutional neural networks, each of which embeds images from different scales or different body parts. The final combined model performs much better than each single model. The experimental results show that our approach achieves very promising results on the challenging CUHK03, CUHK01, and VIPeR data sets. [ABSTRACT FROM PUBLISHER]

Person re-identification is a challenging problem that aims at matching persons across multiple non-overlapping cameras. Previous works on person re-identification mainly focus on solving the problems caused by pose variations, backgrounds, and occlusion while ignoring the viewpoint variations. To address the viewpoint problem, we propose a Viewpoint Transform Matching (VTM) model, which reduces the inference of viewpoints by feature-level viewpoint transformation. Overall, we design a framework by using viewpoint specific branches to separately extract the representation of different viewpoints. Specifically, we first select the branch for an input image, according to its viewpoint information. In the branch, we transform the feature to other viewpoints by establishing a mutually transformable connection between the features of different viewpoints. To suppress the interference among viewpoints, we propose a viewpoint transform classifier module to independently train a classifier for each viewpoint. To improve the effectiveness of the transform, we propose a viewpoint transform loss to guarantee the consistency of the original features and the transformed features. Experiments conducted on Market-1501, DukeMTMC-reID and CUHK03 show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In recent years, the field of person re-identification has made significant advances riding on the wave of deep learning. However, owing to the fact that there are much more easy examples than those meaningful hard examples in a dataset, the training tends to stagnate quickly and the model may suffer from over-fitting, which leads to some error matching of models especially for some hard samples during the test process. Therefore, the hard sample mining method is fateful to optimize the model and improve the learning efficiency. In this paper, an Adaptive Hard Sample Mining algorithm is proposed for training a robust person re-identification model. No need for hand-picking the images in the batch or designing the loss function for both positive and negative pairs, we can briefly calculate the hard level by comparing the prediction result with the true label of the sample. Meanwhile, taking into account the change in the number of samples required for the model during training process, an adaptive threshold of hard level can make the algorithm not only stay in step with training process harmoniously but also alleviate the under-fitting and over-fitting problem simultaneously. Besides, the designed network to implement the approach is very efficient and has good generalization performance that can be combined with various existing models readily. Experimental results on Market-1501, DukeMTMC-reID and CUHK03 datasets clearly demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• Bridging the gaps between detection and tracking task by the D2T framework. • Addressing the different object definition problem by general-to-specific network. • Utilizing the temporal contextual information by introducing a spatial constraint. • Adaptive tracking through an online update strategy. Object detection methods draw increasing attention in deep learning based visual tracking algorithms due to their robust discrimination and powerful regression ability. To further explore the potential of object detection methods in the visual tracking task, there are two gaps that need to be bridged. The first is the difference in object definition. Object detection is class-specific while visual tracking is class-agnostic. Moreover, visual tracking needs to differentiate the target from intra-class distractors. The second is the difference in temporal dimension. Different from object detection which processes still-image, visual tracking concentrates on objects which vary continuously with time. In this paper, we propose a Detection to Tracking (D2T) framework to address the above issues and effectively transfer existing advanced detection methods to visual tracking task. Specifically, to bridge the gap of object definition, we propose a general-to-specific network that separates learning general object features and instance-level features. To make full use of the contextual information while adapting to the appearance variation of targets, we propose a temporal strategy combining short-term constraint and long-term updating. To the best of our knowledge, our D2T framework is the first universal framework which directly transfers deep learning based object detectors to visual tracking task. It provides a novel solution to visual object tracking, and it achieves superior performance in several public datasets. [ABSTRACT FROM AUTHOR]