Search

Light fields suffer from a fundamental resolution tradeoff between the angular and the spatial domain. In this paper, we present a novel cross-scale light field super-resolution approach (up to $\text{8}\times$ resolution gap) to super-resolve low-resolution (LR) light field images that are arranged around a high-resolution (HR) reference image. To bridge the enormous resolution gap between the cross-scale inputs, we introduce an intermediate view denoted as single image super-resolution (SISR) image, i.e., super-resolving LR input via single image based super-resolution scheme, which owns identical resolution as HR image yet lacks high-frequency details that SISR scheme cannot recover under such significant resolution gap. By treating the intermediate SISR image as the low-frequency part of our desired HR image, the remaining issue of recovering high-frequency components can be effectively solved by the proposed high-frequency compensation super-resolution (HCSR) method. Essentially, HCSR works by transferring as much as possible the high-frequency details from the HR reference view to the LR light field image views. Moreover, to solve the nontrivial warping problem that induced by the significant resolution gaps between the cross-scale inputs, we compute multiple disparity maps from the reference image to all the LR light field images, followed by a blending strategy to fuse for a refined disparity map; finally, a high-quality super-resolved light field can be obtained. The superiority of our proposed HCSR method is validated on extensive datasets including synthetic, real-world and challenging microscope scenes.

This paper addresses the problem of disparity map accurate estimation in the cross-scale reference-based light field, which consists several low-quality images arranged around one central high-resolution (HR) image. In the framework, we use a HR image-guidance CNN (HRIG-CNN) for estimating the disparity map in the HR level. Specifically, we first calculate the coarse disparity map using our cross-pattern strategy, which can blend the multiple disparity maps. And then, we refine this coarse disparity map using HRIG-CNN for obtaining high-quality disparity map, which contains detail information and preserve edge information. With the HR image guidance, our HRIG-CNN achieves state-of-the-art for obtaining disparity map in such hybrid light field condition. In the end, we provide both quantitative and qualitative evaluations on different methods, and demonstrate the high performance and robustness of the proposed framework compared with the state-of-the-arts algorithms.

With the development of consumer light field cameras, the light field imaging has become an extensively used method for capturing the three-dimensional appearance of a scene. The depth estimation often requires a dense sampled light field in the angular domain or a high resolution in the spatial domain. However, there is an inherent trade-off between the angular and spatial resolutions of the light field. Recently, some studies for super-resolving the trade-off light field have been introduced. Rather than the conventional approaches that optimize the depth maps, these approaches focus on maximizing the quality of the super-resolved light field. In this article, we investigate how the depth estimation can benefit from these super-resolution methods. Specifically, we compare the qualities of the estimated depth using (a) the original sparse sampled light fields and the reconstructed dense sampled light fields, and (b) the original low-resolution light fields and the high-resolution light fields. Experiment results evaluate the enhanced depth maps using different super-resolution approaches.

We present a novel method to segment an object from multiview images using level set method. Our approach takes advantage of the unique property of level set method in the flexibility of objective energy function design and the adaptability to cut boundary with arbitrary topology. We introduce an iterating optimized 3D level set framework for view coherent segmentation and propose three forces in this framework to drive the convergence of level set to the ideal boundary. In between, the point cloud term and the edge term are designed to give an as-good-as-possible boundary indicator for the level set function, while the local color discriminative classifier is iteratively updated with the multiview silhouette and the 3D point cloud to drive the deformation of the zero level set. Extensive experimental results demonstrate that our approach can produce much more accurate edge localization and more coherent segmentation result across views, compared with the state-of-the-art methods, even for the case of very challenging foreground topologies and ambiguous foreground-background color distribution.