Search

Despite significant progress in machine learning, pedestrian detection in the real-world is still regarded as one of the challenging problems, limited by occluded appearances, cluttered backgrounds, and bad visibility at night. This has caused detection approaches using multi-spectral sensors such as color and thermal which could be complementary to each other. In this paper, we propose a novel sensor fusion framework for detecting pedestrians even in challenging real-world environments. We design a convolutional neural network (CNN) architecture that consists of three-branch detection models taking different modalities as inputs. Unlike existing methods, we consider all detection probabilities from each modality in a unified CNN framework and selectively use them through a channel weighting fusion (CWF) layer to maximize the detection performance. An accumulated probability fusion (APF) layer is also introduced to combine probabilities from different modalities at the proposal-level. We formulate these sub-networks into a unified network, so that it is possible to train the whole network in an end-to-end manner. Our extensive evaluation demonstrates that the proposed method outperforms the state-of-the-art methods on the challenging KAIST, CVC-14, and DIML multi-spectral pedestrian datasets. [ABSTRACT FROM AUTHOR]

Interactive image segmentation has remained an active research topic in image processing and graphics, since the user intention can be incorporated to enhance the performance. It can be employed to mobile devices which now allow user interaction as an input, enabling various applications. Most interactive segmentation methods assume that the initial labels are correctly and carefully assigned to some parts of regions to segment. Inaccurate labels, such as foreground labels in background regions for example, lead to incorrect segments, even by a small number of inaccurate labels, which is not appropriate for practical usage such as mobile application. In this paper, we present an interactive segmentation method that is robust to inaccurate initial labels (scribbles). To address this problem, we propose a structure-aware labeling method using occurrence and co-occurrence probability (OCP) of color values for each initial label in a unified framework. Occurrence probability captures a global distribution of all color values within each label, while co-occurrence one encodes a local distribution of color values around the label. We show that nonlocal regularization together with the OCP enables robust image segmentation to inaccurately assigned labels and alleviates a small-cut problem. We analyze theoretic relations of our approach to other segmentation methods. Intensive experiments with synthetic and manual labels show that our approach outperforms the state of the art. [ABSTRACT FROM AUTHOR]

One aim of detection proposal methods is to reduce the computational overhead of object detection. However, most of the existing methods have significant computational overhead for real-time detection on mobile devices. A fast and accurate proposal method of human detection called personness estimation is proposed, which facilitates real-time human detection on mobile devices and can be effectively integrated into part-based detection, achieving high detection performance at a low computational cost. Our work is based on two observations: (i) normed gradients, which are designed for generic objectness estimation, effectively generate high-quality detection proposals for the person category; (ii) fusing the normed gradients with color attributes improves the performance of proposal generation for human detection. Thus, the candidate windows generated by the personness estimation will very likely contain human subjects. The human detection is then guided by the candidate windows, offering high detection performance even when the detection task terminates prior to completion. This interruptible detection scheme, called anytime detection, enables real-time human detection on mobile devices. Furthermore, we introduce a new evaluation methodology called time-recall curves to practically evaluate our approach. The applicability of our proposed method is demonstrated in extensive experiments on a publicly available dataset and a real mobile device, facilitating acquisition and enhancement of portrait photographs (e.g. selfie) on widespread mobile platforms. [ABSTRACT FROM AUTHOR]

Establishing correspondences is a fundamental task in many image processing and computer vision applications. In particular, finding the correspondences between a non-linearly deformed image pair induced by different modality conditions is a challenging problem. This paper describes a simple but powerful image transform called local area transform (LAT) for modality-robust correspondence estimation. Specifically, LAT transforms an image from the intensity domain to the local area domain, which is invariant under nonlinear intensity deformations, especially radiometric, photometric, and spectral deformations. Experimental results show that LATransformed images provide a consistency for nonlinearly deformed images, even under random intensity deformations. LAT reduces the mean absolute difference by approximately 0.20 and the different pixel ratio by approximately 58% on average, as compared to conventional methods. Furthermore, the reformulation of descriptors with LAT shows superiority to conventional methods, which is a promising result for the tasks of cross-spectral and modality correspondence matching. LAT gains an approximately 23% improvement in the correct detection ratio and a 10% improvement in the recognition rate for the tasks of RGB-NIR cross-spectral template matching and cross-spectral feature matching, respectively. LAT reduces the bad pixel percentage by approximately 15% and the root mean squared errors by 13.5 in the task of cross-radiation stereo matching. LAT also improves the cross-modal dense flow estimation task in terms of warping error, providing 50% error reduction. [ABSTRACT FROM AUTHOR]

Foreground detection is one of the most important and fundamental tasks in many computer vision applications such as real-time video surveillance. Although there have been many efforts to find solutions to this problem, many obstacles such as illumination changes, noises, dynamic backgrounds, and computational complexities have prevented them from being used in real surveillance systems. In this paper, to alleviate these inherent limitations of conventional methods, we propose a fast illumination-robust foreground detection (FIFD) system that provides robustness against illumination variations and noises from various real circumstances with an efficient computational scheme. In contrast to the conventional approaches, our method focuses on efficiently formulating the foreground object detection system by leveraging a foreground candidate region detection and hierarchical distribution map. Specifically, our approach consists of three parts. First, for a query image, foreground candidates are detected by fusing multiple methods. The existence and the block size of the foreground object are determined through the use of the foreground continuity. Second, the foreground block is found from the estimated distribution map and then detected from the extracted valid blocks. Finally, with a labeling scheme, the foreground is detected. To intensively evaluate our approach compared to the conventional methods, we use the publicly available I2R and traffic datasets, and we build a novel electron multiplying charge-coupled device foreground detection benchmark taken in an environment with light lower than 10lux. Experimental results show that our approach provides satisfactory performance compared to the state-of-the-art methods even under very challenging circumstances. Furthermore, our approach is very efficient in that it takes only approximately 31 ms per frame. [ABSTRACT FROM AUTHOR]

A sensor-based vision localization system is one of the most essential technologies in computer vision applications like an autonomous navigation, surveillance, and many others. Conventionally, sensor-based vision localization systems have three inherent limitations, These include, sensitivity to illumination variations, viewpoint variations, and high computational complexity. To overcome these problems, we propose a robust image matching method to provide invariance to the illumination and viewpoint variations by focusing on how to solve these limitations and incorporate this scheme into the vision-based localization system. Based on the proposed image matching method, we design a robust localization system that provides satisfactory localization performance with low computational complexity. Specifically, in order to solve the problem of illumination and viewpoint, we extract a key point using a virtual view from a query image and the descriptor based on the local average patch difference, similar to HC-LBP. Moreover, we propose a key frame selection method and a simple tree scheme for fast image search. Experimental results show that the proposed localization system is four times faster than existing systems, and exhibits better matching performance compared to existing algorithms in challenging environments with difficult illumination and viewpoint conditions. [ABSTRACT FROM AUTHOR]

We propose a novel framework for spatiotemporal action detection using only video-level class labels as weak supervision. Traditional fully-supervised approaches rely on labor-intensive manual annotation of bounding boxes for each frame. In contrast, collecting video-level class labels is significantly less tedious and more feasible compared to annotating frame-level sequences with bounding boxes. To address this challenge, we propose a discriminative action tubelet detector, called DAT-detector, designed to discern discriminative tubelets from action tubelet proposals (ATPs). Whereas the previous approaches have only focused on tubelet selection among the predefined object proposals, our DAT-detector prioritizes the generation of more precise action tubelets using regression and attention modules. Moreover, we introduce an ATP generation method that enhances the quality of tubelet proposals. Our approach achieves state-of-the-art performance on several benchmarks, and also demonstrates competitive performance even with fully-supervised approaches. • We propose a DAT-detector for spatiotemporal action detection using video-level class labels as weak supervision. • The DAT-detector generates precise actions tubes via proposed attention and regression modules. • We enhace tubelet proposal quality with our action tubelet proposal generation method. • Our method significantly outperforms the state-of-the-art action proposal methods. • We achieve remarkable performance in spatiotemporal action detection across multiple benchmarks, effectively competing with fully supervised approaches. [ABSTRACT FROM AUTHOR]

Abstract: We present a novel method for a feature descriptor called an exact order based descriptor (EOD). The proposed method consists of three steps. First, to resolve ordering ambiguity for pixels of the same intensity, an exact order image is created by changing the discrete intensity into a k-dimensional continuous value. Second, exact order based features are generated globally and locally. Finally, the EOD is constructed by combining the global and local exact order features using the discrete cosine transform. Experimental results show that the proposed method outperforms other state-of-the-art descriptors over a number of images. [Copyright &y& Elsevier]

Typically, hierarchical reinforcement learning (RL) requires skills that are applicable to various downstream tasks. Although several recent studies have proposed the supervised and unsupervised learning of such skills, the learned skills are often entangled, which hinders their interpretation. To alleviate this, we propose a novel method to use weak labels for learning disentangled skills from the continuous latent representations of trajectories. To this end, we extended a trajectory variational autoencoder (VAE) to impose an inductive bias using weak labels, which explicitly enforces the disentangling of the trajectory representations into factors of interest intended for the model to learn. Using the latent representations as skills, a skill-based policy network is trained to generate trajectories similar to the learned decoder of the trajectory VAE. Furthermore, using the disentangled skill, we propose a skill repetition that can expand the entire trajectories generated by the policy at test time, resulting in an effective planning strategy. Experiments were performed on several challenging navigation tasks in mazes, and the results demonstrate the effectiveness of our method at solving hierarchical RL problems even with a long horizon and sparse rewards. [Display omitted] • Learning skills as continuous latent variables with disentangled factors. • Using weak labels to train a trajectory variational autoencoder. • Skill repetition to expand the whole trajectory. [ABSTRACT FROM AUTHOR]

Highlights • A self-supervised learning which does not require human annotations for training CNN. • Learning the correct arrangement of object proposals to represent an image by CNN. • Demonstrating the advantage of our model by applying it to PASCAL VOC datasets. • Application to other vision tasks including image retrieval and semantic matching. Abstract Learning visual representations plays an important role in computer vision and machine learning applications. It facilitates a model to understand and perform high-level tasks intelligently. A common approach for learning visual representations is supervised one which requires a huge amount of human annotations to train the model. This paper presents a self-supervised approach which learns visual representations from input images without human annotations. We learn the correct arrangement of object proposals to represent an image using a convolutional neural network (CNN) without any manual annotations. We hypothesize that the network trained for solving this problem requires the embedding of semantic visual representations. Unlike existing approaches that use uniformly sampled patches, we relate object proposals that contain prominent objects and object parts. More specifically, we discover the representation that considers overlap, inclusion, and exclusion relationship of proposals as well as their relative position. This allows focusing on potential objects and parts rather than on clutter. We demonstrate that our model outperforms existing self-supervised learning methods and can be used as a generic feature extractor by applying it to object detection, classification, action recognition, image retrieval, and semantic matching tasks. [ABSTRACT FROM AUTHOR]

A vision based real-time rear obstacle detection system is one of the most essential technologies, which can be used in many applications such as a parking assistance systems and intelligent vehicles. Although disparity is a useful feature for detecting obstacles, estimating a correct disparity map is a hard problem due to the matching ambiguity and noise sensitivity, especially in homogeneous regions. To overcome these problems, we leverage reliable disparities only for obstacle detection. A reliability factor is introduced to measure an inhomogeneity of the regions quantitatively. It is computed at each superpixel to consider the noise sensitivity of pixel-wise gradients and to assign similar reliability value within a same object. It includes two major components: firstly, In a feature extraction and combining stage, we extract three features from stereo images such as disparity, superpixel segments and pixel-wise gradient and compute the reliability of disparity from superpixel segments and the pixel-wise gradient. Secondly, In an obstacle detection stage, a disparity feature with reliability votes for localizing obstacles and dominant candidates in voting map are selected as initial obstacle region. The initial obstacle regions are expanded into their neighbor superpixels based on CIELAB color similarity and distance similarity between superpixels. Experimental results show satisfactory performance under various real parking environments. Its detection rate is at least 4% higher than those of other existing methods, and its false detection rate is more than 10% lower and thus, can be used for parking assistance system. [ABSTRACT FROM AUTHOR]

A vision based rear obstacle detection system is one of the most essential technologies, which can be used in many applications such as a parking assistance systems and intelligent vehicles. Stereo matching based obstacle detection methods have two inherent limitations, including sensitivity to illumination variation and high computational complexity. To overcome these problems, we present a hierarchical census transform (HCT)-based stereo matching method, and proposes a real-time rear obstacle detection system using fisheye stereo cameras. It includes two major components: firstly, we use a hierarchical approach to improve computational efficiency and reduce dependency on a matching window size. Computation time can also be accelerated with compute unified device architecture (CUDA) implementation because we designed the proposed method for parallel processing. Secondly, our method provides more accurate depth information, especially for moving objects, through cost aggregation using color and motion information. Experimental results show satisfactory performance under various real parking environments. The performance of the proposed method is superior to those of the conventional methods in terms of runtime and accuracy of depth estimation. Moreover, correct detection rate of the proposed system is 12.51% better than those of other systems, while its false and miss detection rates are 11.09% and 12.51% lower than those of other detection systems. [ABSTRACT FROM AUTHOR]

[Display omitted] • A novel framework for monocular depth estimation via a student–teacher strategy. • Introducing a data ensemble and stereo confidence-guided regression loss. • Constructing a new large-scale outdoor stereo dataset named the DIML/CVL dataset. • Demonstrating the feature representation of our trained-model for high-level tasks. Current self-supervised methods for monocular depth estimation are largely based on deeply nested convolutional networks that leverage stereo image pairs or monocular sequences during the training phase. However, they often exhibit inaccurate results around occluded regions and depth boundaries. In this paper, we present a simple yet effective approach for monocular depth estimation using stereo image pairs. The study aims to propose a student–teacher strategy in which a shallow student network is trained with the auxiliary information obtained from a deeper and more accurate teacher network. Specifically, we first train the stereo teacher network by fully utilizing the binocular perception of 3-D geometry, and then use the depth predictions of the teacher network to train the student network for monocular depth inference. This enables us to exploit all available depth data from massive unlabeled stereo pairs. We propose a strategy that involves the use of a data ensemble to merge the multiple depth predictions of the teacher network to improve the training samples by collecting non-trivial knowledge beyond a single prediction. To refine the inaccurate depth estimation that is used when training the student network, we further propose stereo confidence guided regression loss that handles the unreliable pseudo depth values in occlusion, texture-less region, and repetitive pattern. To complement the existing dataset comprising outdoor driving scenes, we built a novel large-scale dataset consisting of one million outdoor stereo images taken using hand-held stereo cameras. Finally, we demonstrate that the monocular depth estimation network provides feature representations that are suitable for high-level vision tasks. The experimental results for various outdoor scenarios demonstrate the effectiveness and flexibility of our approach, which outperforms state-of-the-art approaches. [ABSTRACT FROM AUTHOR]