Search

Consumer-grade RGB-D cameras, such as Kinect sensors, can provide support for much more real-time tasks of 3-D vision than game controllers. However, the inherent depth degradations caused by their infrared ranging will constrain their application potential, but can hardly be avoided through the improvement of the sensor design. Therefore, in this paper, we proposed a contours-guided shape-adaptive morphology filter to efficiently recover the depth of Kinect sensors. First, we put forward a statistical concept to quantitatively evaluate the texture richness of imaging sensors’ data and verify the applicability of morphology filtering on both Kinect 1 and 2 depth data. Then, considering the significance of the semantic contours, a multiresolution RGB-D contour extraction method is introduced to suppress the texture inside objects. Therewith, shape-adaptive structuring element (SASE) for each missing or untrusted depth pixel is created in terms of the contour guidance and the feature of human visual system. Efficient and accurate depth recovery can be finally achieved by combining morphology filtering and the obtained SASEs. Experiments on simulated data set, real Kinect 1, and Kinect 2 data show that our method performs better than many competing state-of-the-art approaches, and avoids the blurring around depth discontinuities.

In 2D face recognition, result may suffer from the impact of varying pose, expression, and illumination conditions. However, 3D face recognition utilizes depth information to enhance systematic robustness. Thus, an improved deep convolutional neural network (DCNN) combined with softmax classifier to identify face is trained. First, the preprocessing of color image and depth map is different in removing redundant information. Then, the feature extraction networks for 2D face image and depth map are, respectively, build with the principle of recognition rate maximization, and parameters about neural networks reset by a series of tests, in order to acquire higher recognition rate. At last, the fusion of two feature layers is the final input of artificial neural network (ANN) recognition system, which is followed by a 64-way softmax output. Experimental results demonstrate that it is effective in improving recognition rate.

Recovering hyperspectral image (HSI) from mixed noise degradation is a challenging and promising theme in remote sensing, particularly when stripes and deadlines exist in several contiguous bands. This paper proposes a HSI’s restoration method making use of adaptive morphological filtering (AMF) and fusing structure information of an auxiliary color image. An adaptive structuring element (ASE) indicating morphological features of each pixel is generated through information fusion, to simultaneously remove the mixed noise and preserve fine spatial structures. This key technology contains three main steps. First, edges are extracted from the auxiliary image exploiting its color information; then, an edge-constraint growing algorithm is used to generate the clustering kernel; finally, the ASE is obtained via goal-guided k-means clustering. The ASE has extensive application value, for it can be an enhancing module for most filters-based restoration methods, to mitigate the structural damage due to the fixed mask. Among these methods, Gaussian filter for preprocessing and majority voting for postprocessing are introduced in this paper as representatives. In addition, the auxiliary image can be both visible image of multisensor and false RGB component of the undamaged bands of the HSI, so it is relatively available. Experiments on simulated and real data sets show obvious effects on denoising and destriping both subjectively and objectively. The advantage of ASE on structure details preserving, compared to conventional approaches, is clearly demonstrated. The application value of the proposed restoration frame and ASE is further proved through the decision-level postprocessing experiments.

Simultaneously recovering hyperspectral images (HSIs) from mixed degradations is a classical inverse problem, which has attracted major research efforts. Low-rank matrix recovery (LRMR) has been proved to be an effective method. This paper proposes a multi-scale recovering model based on 3D Gaussian pyramid decomposition and residual reconstitution to improve the LRMR on its adaptability of local correlative noises (including stripes, dead lines and impulse noise) removal. By compressing the HSI cube to lower-resolution layers in spatial domain, the non-local low rank property of clean HSI can be better utilized. LRMR algorithm is applied from the top. The following layers are reconstituted with the upper recovery result and their original decomposition residual, and then executed LRMR until the bottom of the pyramid. In the proposed procedure, mixed noises are removed progressively in terms of frequency, besides the details of clean HSI are well preserved. Experimental results on simulated and real data in terms of qualitative and quantitative assessments show significant improvements over conventional methods.

Unmixing based fusion aims at generating a high spectral-spatial resolution image (HSS) with the same surface features of the high spatial resolution multispectral image (MS) and low spatial resolution hyperspectral image (HS). In this paper, a new fusion method is proposed to improve the fusion performance by taking further advantage of the distribution characteristics of ground objects. First, we put forward a local adaptive sparse unmixing based fusion (LASUF) algorithm, in which the sparsity of the abundance matrices is appended as the constraint to the optimization fusion, considering the limited categories of ground objects in a specific range and the local correlation of their distribution. Then, to correct the possible original subpixel misregistrations or those introduced by the fusion procedures, a subpixel calibration method based on optimal matching adaptive morphology filtering (OM-AMF) is designed. Experiments on various datasets captured by different sensors demonstrate that the proposed fusion algorithm surpasses other typical fusion techniques in both spatial and spectral domains. The proposed method effectively preserves the spectral composition features of the isolated ground objects within a small area. In addition, the OM-AMF postprocessing is able to spatially correct the fusion results at a subpixel level and preserve the spectral features simultaneously.

A quaternion vector gradient filter is proposed for RGB-depth (RGB-D) video contour detection. First, a holistic quaternion vector system is introduced to synthetically express the color and depth information, by adding the depth to its scalar part. Then, a convolution differential operator for quaternion vector is proposed to highlight edges with both depth and chromatic variations but restrain the gradient of intensity term. In addition, the quaternion vector gradients are adaptively weighted utilizing depth confidence measure and the quadtree decomposition of the coding tree units in the video streaming. Results on the 3-D high-efficiency video coding test sequences and quantitative simulated experiments on Berkeley segmentation datasets both indicate the availability of the proposed gradient-based method on detecting the semantic contour of the RGB-D videos.