Search

Consumer-grade RGB-D cameras, such as Kinect sensors, can provide support for much more real-time tasks of 3-D vision than game controllers. However, the inherent depth degradations caused by their infrared ranging will constrain their application potential, but can hardly be avoided through the improvement of the sensor design. Therefore, in this paper, we proposed a contours-guided shape-adaptive morphology filter to efficiently recover the depth of Kinect sensors. First, we put forward a statistical concept to quantitatively evaluate the texture richness of imaging sensors’ data and verify the applicability of morphology filtering on both Kinect 1 and 2 depth data. Then, considering the significance of the semantic contours, a multiresolution RGB-D contour extraction method is introduced to suppress the texture inside objects. Therewith, shape-adaptive structuring element (SASE) for each missing or untrusted depth pixel is created in terms of the contour guidance and the feature of human visual system. Efficient and accurate depth recovery can be finally achieved by combining morphology filtering and the obtained SASEs. Experiments on simulated data set, real Kinect 1, and Kinect 2 data show that our method performs better than many competing state-of-the-art approaches, and avoids the blurring around depth discontinuities.

Recovering hyperspectral image (HSI) from mixed noise degradation is a challenging and promising theme in remote sensing, particularly when stripes and deadlines exist in several contiguous bands. This paper proposes a HSI’s restoration method making use of adaptive morphological filtering (AMF) and fusing structure information of an auxiliary color image. An adaptive structuring element (ASE) indicating morphological features of each pixel is generated through information fusion, to simultaneously remove the mixed noise and preserve fine spatial structures. This key technology contains three main steps. First, edges are extracted from the auxiliary image exploiting its color information; then, an edge-constraint growing algorithm is used to generate the clustering kernel; finally, the ASE is obtained via goal-guided k-means clustering. The ASE has extensive application value, for it can be an enhancing module for most filters-based restoration methods, to mitigate the structural damage due to the fixed mask. Among these methods, Gaussian filter for preprocessing and majority voting for postprocessing are introduced in this paper as representatives. In addition, the auxiliary image can be both visible image of multisensor and false RGB component of the undamaged bands of the HSI, so it is relatively available. Experiments on simulated and real data sets show obvious effects on denoising and destriping both subjectively and objectively. The advantage of ASE on structure details preserving, compared to conventional approaches, is clearly demonstrated. The application value of the proposed restoration frame and ASE is further proved through the decision-level postprocessing experiments.

The depth map obtained by Microsoft Kinect is often accompanied with a large area information loss called black hole. In this paper, an optimal algorithm for image restoration with multi-resolution anisotropic diffusion is proposed to fill the black areas. At the same times, the restriction of anisotropic diffusion algorithm can be broken through by using multi-resolution. Using the color map as a guidance to refine the details of the image. Then according to local adjustment, the error rate can be effectively reduced. At the end of this paper, the joint bilateral filter (JBF) is introduced as a contrast, and the PCL open source point cloud database is used for 3D reconstruction. Compared with competing methods, the proposed algorithm can fill the larger black holes significantly. The experiments show that it also has good performance on reducing the error rate and preserving the edge details.

In 2D face recognition, result may suffer from the impact of varying pose, expression, and illumination conditions. However, 3D face recognition utilizes depth information to enhance systematic robustness. Thus, an improved deep convolutional neural network (DCNN) combined with softmax classifier to identify face is trained. First, the preprocessing of color image and depth map is different in removing redundant information. Then, the feature extraction networks for 2D face image and depth map are, respectively, build with the principle of recognition rate maximization, and parameters about neural networks reset by a series of tests, in order to acquire higher recognition rate. At last, the fusion of two feature layers is the final input of artificial neural network (ANN) recognition system, which is followed by a 64-way softmax output. Experimental results demonstrate that it is effective in improving recognition rate.

3D High Efficiency Video Coding (3D-HEVC) provides a significant potential on increasing the compression ratio of multi-view RGB-D videos. However, the bit rate still rises dramatically with the improvement of the video resolution, which will bring challenges to the transmission network, especially the mobile network. This paper propose an ROI multi-resolution compression method for 3D-HEVC to better preserve the information in ROI on condition of limited bandwidth. This is realized primarily through ROI extraction and compression multi-resolution preprocessed video as alternative data according to the network conditions. At first, the semantic contours are detected by the modified structured forests to restrain the color textures inside objects. The ROI is then determined utilizing the contour neighborhood along with the face region and foreground area of the scene. Secondly, the RGB-D videos are divided into slices and compressed via 3D-HEVC under different resolutions for selection by the audiences and applications. Afterwards, the reconstructed low-resolution videos from 3D-HEVC encoder are directly up-sampled via Laplace transformation and used to replace the non-ROI areas of the high-resolution videos. Finally, the ROI multi-resolution compressed slices are obtained by compressing the ROI preprocessed videos with 3D-HEVC. The temporal and special details of non-ROI are reduced in the low-resolution videos, so the ROI will be better preserved by the encoder automatically. Experiments indicate that the proposed method can keep the key high-frequency information with subjective significance while the bit rate is reduced.

Efficiently transmitting the extra information of three dimensional (3D) video is becoming a key issue of the development of 3DTV. 2D plus depth format not only occupies the smaller bandwidth and is compatible transmission under the condition of the existing channel, but also can provide technique support for advanced 3D video compression in some extend. This paper proposes an ROI-preserving compression scheme to further improve the visual quality at a limited bit rate. According to the connection between the focus of Human Visual System (HVS) and depth information, region of interest (ROI) can be automatically selected via depth map progressing. The main improvement from common method is that a meanshift based segmentation is executed to the depth map before foreground ROI selection to keep the integrity of scene. Besides, the sensitive areas along the edges are also protected. The Spatio-temporal filtering adapting to H.264 is used to the non-ROI of both 2D video and depth map before compression. Experiments indicate that, the ROI extracted by this method is more undamaged and according with subjective feeling, and the proposed method can keep the key high-frequency information more effectively while the bit rate is reduced.

A quaternion vector gradient filter is proposed for RGB-depth (RGB-D) video contour detection. First, a holistic quaternion vector system is introduced to synthetically express the color and depth information, by adding the depth to its scalar part. Then, a convolution differential operator for quaternion vector is proposed to highlight edges with both depth and chromatic variations but restrain the gradient of intensity term. In addition, the quaternion vector gradients are adaptively weighted utilizing depth confidence measure and the quadtree decomposition of the coding tree units in the video streaming. Results on the 3-D high-efficiency video coding test sequences and quantitative simulated experiments on Berkeley segmentation datasets both indicate the availability of the proposed gradient-based method on detecting the semantic contour of the RGB-D videos.