Search

Object detection in aerial images has achieved remarkable progress with the advent of deep convolutional neural networks (CNNs). It is, however, still a challenging task since the objects in aerial images are arbitrarily oriented and often densely packed. In this letter, we propose a novel method for oriented object detection in aerial images that represents objects as rotation equivariant semantic keypoints. Unlike conventional methods that represent object rotation according to angles from each axis in the Cartesian coordinate system, we represent object using a canonical orientation to ensure rotation equivariance. We accomplish this by representing an object as semantic keypoints, where each keypoint of the object consistently corresponds to the semantic part, regardless of rotation variation. To this end, we define the “head” point of the object as the canonical orientation and the remaining bounding box vectors as semantic keypoints in clockwise order. To discriminate visual attributes between different categories, we further use category-specific semantic keypoints, so that object classification and localization can be jointly solved in a cooperative manner. Our experiments demonstrate the effectiveness of rotation equivariant semantic keypoints on oriented object detection. [ABSTRACT FROM AUTHOR]

This paper presents a deep architecture, called pyramidal semantic correspondence networks (PSCNet), that estimates locally-varying affine transformation fields across semantically similar images. To deal with large appearance and shape variations that commonly exist among different instances within the same object category, we leverage a pyramidal model where the affine transformation fields are progressively estimated in a coarse-to-fine manner so that the smoothness constraint is naturally imposed. Different from the previous methods which directly estimate global or local deformations, our method first starts to estimate the transformation from an entire image and then progressively increases the degree of freedom of the transformation by dividing coarse cell into finer ones. To this end, we propose two spatial pyramid models by dividing an image in a form of quad-tree rectangles or into multiple semantic elements of an object. Additionally, to overcome the limitation of insufficient training data, a novel weakly-supervised training scheme is introduced that generates progressively evolving supervisions through the spatial pyramid models by leveraging a correspondence consistency across image pairs. Extensive experimental results on various benchmarks including TSS, Proposal Flow-WILLOW, Proposal Flow-PASCAL, Caltech-101, and SPair-71k demonstrate that the proposed method outperforms the lastest methods for dense semantic correspondence. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of single image de-raining, that is, the task of recovering clean and rain-free background scenes from a single image obscured by a rainy artifact. Although recent advances adopt real-world time-lapse data to overcome the need for paired rain-clean images, they are limited to fully exploit the time-lapse data. The main cause is that, in terms of network architectures, they could not capture long-term rain streak information in the time-lapse data during training owing to the lack of memory components. To address this problem, we propose a novel network architecture combining the time-lapse data and, the memory network that explicitly helps to capture long-term rain streak information. Our network comprises the encoder-decoder networks and a memory network. The features extracted from the encoder are read and updated in the memory network that contains several memory items to store rain streak-aware feature representations. With the read/update operation, the memory network retrieves relevant memory items in terms of the queries, enabling the memory items to represent the various rain streaks included in the time-lapse data. To boost the discriminative power of memory features, we also present a novel background selective whitening (BSW) loss for capturing only rain streak information in the memory network by erasing the background information. Experimental results on standard benchmarks demonstrate the effectiveness and superiority of our approach. [ABSTRACT FROM AUTHOR]

Stereo matching aiming to perceive the 3-D geometry of a scene facilitates numerous computer vision tasks used in advanced driver assistance systems (ADAS). Although numerous methods have been proposed for this task by leveraging deep convolutional neural networks (CNNs), stereo matching still remains an unsolved problem due to its inherent matching ambiguities. To overcome these limitations, we present a method for jointly estimating disparity and confidence from stereo image pairs through deep networks. We accomplish this through a minmax optimization to learn the generative cost aggregation networks and discriminative confidence estimation networks in an adversarial manner. Concretely, the generative cost aggregation networks are trained to accurately generate disparities at both confident and unconfident pixels from an input matching cost that are indistinguishable by the discriminative confidence estimation networks, while the discriminative confidence estimation networks are trained to distinguish the confident and unconfident disparities. In addition, to fully exploit complementary information of matching cost, disparity, and color image in confidence estimation, we present a dynamic fusion module. Experimental results show that this model outperforms the state-of-the-art methods on various benchmarks including real driving scenes. [ABSTRACT FROM AUTHOR]

Leveraging on recent advances in deep convolutional neural networks (CNNs), single image deraining has been studied as a learning task, achieving an outstanding performance over traditional hand-designed approaches. Current CNNs based deraining approaches adopt the supervised learning framework that uses a massive training data generated with synthetic rain streaks, having a limited generalization ability on real rainy images. To address this problem, we propose a novel learning framework for single image deraining that leverages time-lapse sequences instead of the synthetic image pairs. The deraining networks are trained using the time-lapse sequences in which both camera and scenes are static except for time-varying rain streaks. Specifically, we formulate a background consistency loss such that the deraining networks consistently generate the same derained images from the time-lapse sequences. We additionally introduce two loss functions, the structure similarity loss that encourages the derained image to be similar with an input rainy image and the directional gradient loss using the assumption that the estimated rain streaks are likely to be sparse and have dominant directions. To consider various rain conditions, we leverage a dynamic fusion module that effectively fuses multi-scale features. We also build a novel large-scale time-lapse dataset providing real world rainy images containing various rain conditions. Experiments demonstrate that the proposed method outperforms state-of-the-art techniques on synthetic and real rainy images both qualitatively and quantitatively. On the high-level vision tasks under severe rainy conditions, it has been shown that the proposed method can be utilized as a pre-preprocessing step for subsequent tasks. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have facilitated substantial progress on various problems in computer vision and image processing. However, applying them to image fusion has remained challenging due to the lack of the labelled data for supervised learning. This paper introduces a deep image fusion network (DIF-Net), an unsupervised deep learning framework for image fusion. The DIF-Net parameterizes the entire processes of image fusion, comprising of feature extraction, feature fusion, and image reconstruction, using a CNN. The purpose of DIF-Net is to generate an output image which has an identical contrast to high-dimensional input images. To realize this, we propose an unsupervised loss function using the structure tensor representation of the multi-channel image contrasts. Different from traditional fusion methods that involve time-consuming optimization or iterative procedures to obtain the results, our loss function is minimized by a stochastic deep learning solver with large-scale examples. Consequently, the proposed method can produce fused images that preserve source image details through a single forward network trained without reference ground-truth labels. The proposed method has broad applicability to various image fusion problems, including multi-spectral, multi-focus, and multi-exposure image fusions. Quantitative and qualitative evaluations show that the proposed technique outperforms existing state-of-the-art approaches for various applications. [ABSTRACT FROM AUTHOR]

We present a deep architecture and learning framework for establishing correspondences across cross-spectral visible and infrared images in an unpaired setting. To overcome the unpaired cross-spectral data problem, we design the unified image translation and feature extraction modules to be learned in a joint and boosting manner. Concretely, the image translation module is learned only with the unpaired cross-spectral data, and the feature extraction module is learned with an input image and its translated image. By learning two modules simultaneously, the image translation module generates the translated image that preserves not only the domain-specific attributes with separate latent spaces but also the domain-agnostic contents with feature consistency constraint. In an inference phase, the cross-spectral feature similarity is augmented by intra-spectral similarities between the features extracted from the translated images. Experimental results show that this model outperforms the state-of-the-art unpaired image translation methods and cross-spectral feature descriptors on various visible and infrared benchmarks. [ABSTRACT FROM AUTHOR]

Dragging a rectangular box is arguably one of the most intuitive and popular user interfaces to select a whole object. However, the conventional dragging method cannot be easily applied on touch-screen devices. This paper proposes a more convenient and lazier way of bounding-box (BB) drawing for touch-screen devices. The proposed method of BB drawing, termed Lazy Dragging, is designed to yield accurate BB results at interactive speeds. To this end, Lazy Dragging proceeds in two stages: (i) it first filters out explicit non-borders via graph-based segmentation; (ii) it then singles out the best four box sides among the remaining candidates using edge feature-based random forest. In this second stage, global and local edge-based border detection and a combined approach of these global and local schemes are utilized. This paper also proposes an optional stage to simplify and ameliorate further user refinement by providing invisible magnetic guidelines. This technique guides a user's touch to the nearest superpixel boundaries. Extensive experiments on a real-world dataset demonstrate that Lazy Dragging convincingly enhances the quality of input BBs in quasi real-time, enabling effortless object selection on small touch-screen devices. [ABSTRACT FROM PUBLISHER]

The human category is among the most commonly captured and therefore important subject in digital cameras. As a result, human segmentation also plays an important role in the camera system. This paper presents a framework of non-parametric learning that achieves human segmentation using support vector machine (SVM). In the training stage, the training human windows are warped to a normalized size and oversegmented into regular superpixels. A two-dimensional (2D) array of SVM models is then trained by extracting various edge and color features from each superpixel. In this process of SVM training, the 2D array of SVM models automatically and effectively learns the characteristics of the human shape. Given an affinely warped human window for testing, the proposed method calculates superpixels' initial scores of belonging to a human (foreground) using the trained SVM models. Finally, the initial prediction scores are effectively propagated by optimizing a well-defined energy function using an estimated confidence map. In experiments on a publicly available challenging dataset, the proposed framework rapidly yields excellent results in human segmentation both qualitatively and quantitatively1. [ABSTRACT FROM PUBLISHER]

In 3DTV, it is known that video captions and graphics should be inserted at proper scene depth positions, to prevent possible viewing discomfort. We propose a character generator that automatically analyzes the scene disparities and determines the proper depth of the graphic object to be inserted. The challenge is that the disparity range estimation from feature correspondences is severely affected even by a few outliers, whereas naıuml;ve SURF or BRIEF matching produces a considerable amount of outliers. We propose a multiple-hypothesis feature matching algorithm that considers the disparity coherence between adjacent features, with which most mismatches can be removed according to the reliability aggregated from the neighboring features. To estimate the accurate disparity range from the feature correspondences, a disparity histogram is computed and filtered by a space-time kernel to suppress the effect of incorrect disparities. We also propose the disparity-depth conversion in the asymmetric view frustum which is used for the stereoscopic rendering of graphics. Experimental results show that a 3D graphic object is successfully inserted at the desired depth which is obtained from the proposed disparity range estimation and disparity-depth conversion. [ABSTRACT FROM AUTHOR]

Abstract: We present a novel method for a feature descriptor called an exact order based descriptor (EOD). The proposed method consists of three steps. First, to resolve ordering ambiguity for pixels of the same intensity, an exact order image is created by changing the discrete intensity into a k-dimensional continuous value. Second, exact order based features are generated globally and locally. Finally, the EOD is constructed by combining the global and local exact order features using the discrete cosine transform. Experimental results show that the proposed method outperforms other state-of-the-art descriptors over a number of images. [Copyright &y& Elsevier]

A vision based real-time rear obstacle detection system is one of the most essential technologies, which can be used in many applications such as a parking assistance systems and intelligent vehicles. Although disparity is a useful feature for detecting obstacles, estimating a correct disparity map is a hard problem due to the matching ambiguity and noise sensitivity, especially in homogeneous regions. To overcome these problems, we leverage reliable disparities only for obstacle detection. A reliability factor is introduced to measure an inhomogeneity of the regions quantitatively. It is computed at each superpixel to consider the noise sensitivity of pixel-wise gradients and to assign similar reliability value within a same object. It includes two major components: firstly, In a feature extraction and combining stage, we extract three features from stereo images such as disparity, superpixel segments and pixel-wise gradient and compute the reliability of disparity from superpixel segments and the pixel-wise gradient. Secondly, In an obstacle detection stage, a disparity feature with reliability votes for localizing obstacles and dominant candidates in voting map are selected as initial obstacle region. The initial obstacle regions are expanded into their neighbor superpixels based on CIELAB color similarity and distance similarity between superpixels. Experimental results show satisfactory performance under various real parking environments. Its detection rate is at least 4% higher than those of other existing methods, and its false detection rate is more than 10% lower and thus, can be used for parking assistance system. [ABSTRACT FROM AUTHOR]