Search

The occlusion problem is a major challenge in the field of computer vision. Synthetic aperture imaging (SAI) is often used for surface reconstruction of occluded objects. However, SAI usually relies on high-speed information transmission devices. In addition, a large amount of information in the scene is lost, when handling low-resolution input images. This limitation results in unclear reconstructed regions in the synthetic aperture image and thus hinders the application of SAI in downstream tasks. We propose a multi-view super-resolution SAI method. It aims to generate high-resolution synthetic aperture images using images acquired by few of low-resolution acquisition devices. The main contributions of this paper are: (1) a multi-view super-resolution algorithm is proposed. It can generate clear synthetic aperture images in an array with a limited number of cameras. (2) By exploiting the correlation between views, the proposed algorithm can generate super-resolution synthetic aperture images with more accurate image structure and sharper image edges. (3) A feature extraction module is proposed. It can effectively extract the complementary relationship between pictures from different perspectives. The experimental results show that the proposed method can generate a reconstructed image of the occluded object surface with clear edges and accurate structure. Compared to conventional SAI, our method improves 5.7 % / 21.1 % on peak signal-to-noise ratio (PSNR)/structure similarity index measure (SSIM) and 4.4 % / 9.2 % on PSNR/SSIM respectively on two datasets compared to other state-of-the-art super-resolution methods. [ABSTRACT FROM AUTHOR]

“Seeing through” occluders is one of the most important effects that can be achieved with synthetic aperture imaging. As well, the occlusion problem, a challenging task for many computer vision applications, can be easily handled. Synthetic aperture imaging takes advantage of the property that only objects on the focal plane are sharp. The resulting image that is obtained by averaging images from different views consists of blurry objects away from the focal plane and sharp objects on the focal plane. Removing the blurriness caused by defocusing in synthetic aperture images to achieve an all-in-focus “seeing through” image is a challenging research problem. In this paper, we propose a novel method to improve the image quality of synthetic aperture imaging using image matting via energy minimization by estimating the foreground and the background. In particular, we first estimate the out-of-focus region by focusing on the background objects in each camera view using energy minimization. Next, we utilize a labeling method to create a sharp “see through” synthetic aperture image of the hidden objects. Then, image matting is used to extract the alpha matte of the hidden objects. Finally, by compositing the hidden objects with the estimated background regions, a sharp “see through” synthetic aperture image is created. The experimental results show that the proposed method outperforms the traditional synthetic aperture imaging method [1] as well as its improved versions [2]–[4], which simply dim and blur the area in the image that is out of focus, and a recent all-in-focus method [5]. We show that both the occluded objects and the background can be combined using our method to create a sharp synthetic aperture image. [ABSTRACT FROM PUBLISHER]

Abstract: Synthetic aperture imaging using an array of cameras, which has become popular recently, can easily handle the occlusion problem by “seeing through” occluders. Unfortunately, the resulting image is still blurry because it combines information not only from the region of interest but also from the occluding regions. Removing the blurriness of synthetic aperture images has become a challenging task for many computer vision applications. In this paper, we propose a novel method to improve the image quality of synthetic aperture imaging using energy minimization. Unlike the conventional synthetic aperture imaging method, which averages images from all the camera views, we reformulate the problem as a labeling problem. In particular, we use the energy minimization method to label each pixel in each camera view to decide whether or not it belongs to an occluder. After that, the focusing at the desired depth is by averaging pixels that are not labeled as occluder. The experimental results show that the proposed method outperforms the traditional synthetic aperture imaging method as well as its improved versions, which are simply dim and blur occluders in the resulting image. To the best of our knowledge, our proposed method is the first one for improving the results of synthetic aperture image without using a training set from the input sequence. As well, it is the first method that makes no assumptions on whether or not the objects in the scenes are static. [Copyright &y& Elsevier]

Abstract: This paper proposes a novel multi-object detection method using multiple cameras. Unlike conventional multi-camera object detection methods, our method detects multiple objects using a linear camera array. The array can stream different views of the environment and can be easily reconfigured for a scene compared with the overhead surround configuration. Using the proposed method, the synthesized results can provide not only views of significantly occluded objects but also the ability of focusing on the target while blurring objects that are not of interest. Our method does not need to reconstruct the 3D structure of the scene, can accommodate dynamic background, is able to detect objects at any depth using a new synthetic aperture imaging method based on a simple shift transformation, and can see through occluders. The experimental results show that the proposed method has a good performance and can synthesize objects located within any designated depth interval with much better clarity than that using an existing method. To our best knowledge, it is the first time that such a method using synthetic aperture imaging has been proposed and developed for multi-object detection in a complex scene with a significant occlusion at different depths. [Copyright &y& Elsevier]

• The first time to address the missing information problem caused by heavy occlusion in Synthetic Aperture Imaging. • Our method generates a realistic all-in-focus synthetic aperture image where the information of the occluded region is completely restored. • Extensive experiments on public datasets and our own datasets demonstrate the superior performance of the proposed method over state-of-the-art Synthetic Aperture Imaging methods. Occlusions handling poses a significant challenge to many computer vision and pattern recognition applications. Recently, Synthetic Aperture Imaging (SAI), which uses more than two cameras, is widely applied to reconstruct occluded objects in complex scenes. However, it usually fails in cases of heavy occlusions, in particular, when the occluded information is not captured by any of the camera views. Hence, it is a challenging task to generate a realistic all-in-focus synthetic aperture image which shows a completely occluded object. In this paper, semantic inpainting using a Generative Adversarial Network (GAN) is proposed to address the above-mentioned problem. The proposed method first computes a synthetic aperture image of the occluded objects using a labeling method, and an alpha matte of the partially occluded objects. Then, it uses energy minimization to reconstruct the background by focusing on the background depth of each camera. Finally, the occluded regions of the synthesized image are semantically inpainted using a GAN and the results are composited with the reconstructed background to generate a realistic all-in-focus image. The experimental results demonstrate that the proposed method can handle heavy occlusions and can produce better all-in-focus images than other state-of-the-art methods. Compared with traditional labeling methods, our method can quickly generate label for occlusion without introducing noise. To the best of our knowledge, our method is the first to address missing information caused by heavy occlusions in SAI using a GAN. [ABSTRACT FROM AUTHOR]