Search

In everyday photography, physical limitations of camera sensors and lenses frequently lead to a variety of degradations in captured images such as saturation or defocus blur. A common approach to overcome these limitations is to resort to image stack fusion, which involves capturing multiple images with different focal distances or exposures. For instance, to obtain an all-in-focus image, a set of multi-focus images is captured. Similarly, capturing multiple exposures allows for the reconstruction of high dynamic range. In this paper, we present a novel approach that combines neural fields with an expressive camera model to achieve a unified reconstruction of an all-in-focus high-dynamic-range image from an image stack. Our approach is composed of a set of specialized implicit neural representations tailored to address specific sub-problems along our pipeline: We use neural implicits to predict flow to overcome misalignments arising from lens breathing, depth, and all-in-focus images to account for depth of field, as well as tonemapping to deal with sensor responses and saturation - all trained using a physically inspired supervision structure with a differentiable thin lens model at its core. An important benefit of our approach is its ability to handle these tasks simultaneously or independently, providing flexible post-editing capabilities such as refocusing and exposure adjustment. By sampling the three primary factors in photography within our framework (focal distance, aperture, and exposure time), we conduct a thorough exploration to gain valuable insights into their significance and impact on overall reconstruction quality. Through extensive validation, we demonstrate that our method outperforms existing approaches in both depth-from-defocus and all-in-focus image reconstruction tasks. Moreover, our approach exhibits promising results in each of these three dimensions, showcasing its potential to enhance captured image quality and provide greater control in post-processing. [ABSTRACT FROM AUTHOR]

The spectral and depth (SAD) imaging method plays an important role in the field of computer vision. However, accurate depth estimation and spectral image capture from a single image without increasing the volume of the imaging sensor is still an unresolved problem. Our research finds that a snapshot narrow band imaging (SNBI) method can discern wavelength-dependent spectral aberration and simultaneously capture spectral-aberration defocused images for quantitative depth estimation. First, a micro 4D imaging (M4DI) sensor is proposed by integrating a mono-chromatic imaging sensor with a miniaturized narrow-band microarrayed spectral filter mosaic. The appearance and volume of the M4DI sensor are the same as the integrated mono-chromatic imaging sensor. A simple remapping algorithm was developed to separate the raw image into four narrow spectral band images. Then, a depth estimation algorithm is developed to generate 3D data with a dense depth map at every exposure of the M4DI sensor. Compared with existing SAD imaging method, the M4DI sensor has the advantages of simple implementation, low computational burden, and low cost. A proof-of-principle M4DI sensor was applied to sense the depth of objects and to track a tiny targets trajectory. The relative error in the three-dimensional positioning is less than 7% for objects within 1.1 to 2.8 m. [ABSTRACT FROM AUTHOR]

Background and Objectives: Depth from defocus and defocus deblurring from a single image are two challenging problems caused by the finite depth of field in conventional cameras. Coded aperture imaging is a branch of computational imaging, which is used to overcome these two problems. Up to now, different methods have been proposed for improving the results of either defocus deblurring or depth estimation. In this paper, an asymmetric coded aperture is proposed which improves results of depth estimation and defocus deblurring from a single input image. Methods: To this aim, a multi-objective optimization function taking into consideration both deblurring results and depth discrimination ability is proposed. Since aperture throughput affects on image quality, our optimization function is defined based on illumination conditions and camera specifications which yields an optimized throughput aperture. Because the designed pattern is asymmetric, defocused objects on two sides of the focal plane can be distinguished. Depth estimation is performed using a new algorithm, which is based on perceptual image quality assessment criteria and can discern blurred objects lying in front or behind the focal plane. Results: Extensive simulations as well as experiments on a variety of real scenes are conducted to compare the performance of our aperture with previously proposed ones. Conclusion: Our aperture has been designed for indoor illumination setting. However, the proposed method can be utilized for designing and evaluating appropriate aperture patterns for different imaging conditions. [ABSTRACT FROM AUTHOR]

Multi-view stereo remains a popular choice when recovering 3D geometry, despite performance varying dramatically according to the scene content. Moreover, typical pinhole camera assumptions fail in the presence of shallow depth of field inherent to macro-scale scenes; limiting application to larger scenes with diffuse reflectance. However, the presence of defocus blur can itself be considered a useful reconstruction cue, particularly in the presence of view-dependent materials. With this in mind, we explore the complimentary nature of stereo and defocus cues in the context of multi-view 3D reconstruction; and propose a complete pipeline for scene modelling from a finite aperature camera that encompasses image formation, camera calibration and reconstruction stages. As part of our evaluation, an ablation study reveals how each cue contributes to the higher performance observed over a range of complex materials and geometries. Though of lesser concern with large apertures, the effects of image noise are also considered. By introducing pre-trained deep feature extraction into our cost function, we show a step improvement over per-pixel comparisons; as well as verify the cross-domain applicability of networks using largely in-focus training data applied to defocused images. Finally, we compare to a number of modern multi-view stereo methods, and demonstrate how the use of both cues leads to a significant increase in performance across several synthetic and real datasets. [ABSTRACT FROM AUTHOR]

The spectral and depth (SAD) imaging method plays an important role in the field of computer vision. However, accurate depth estimation and spectral image capture from a single image without increasing the volume of the imaging sensor is still an unresolved problem. Our research finds that a snapshot narrow band imaging (SNBI) method can discern wavelength-dependent spectral aberration and simultaneously capture spectral-aberration defocused images for quantitative depth estimation. First, a micro 4D imaging (M4DI) sensor is proposed by integrating a mono-chromatic imaging sensor with a miniaturized narrow-band microarrayed spectral filter mosaic. The appearance and volume of the M4DI sensor are the same as the integrated mono-chromatic imaging sensor. A simple remapping algorithm was developed to separate the raw image into four narrow spectral band images. Then, a depth estimation algorithm is developed to generate 3D data with a dense depth map at every exposure of the M4DI sensor. Compared with existing SAD imaging method, the M4DI sensor has the advantages of simple implementation, low computational burden, and low cost. A proof-of-principle M4DI sensor was applied to sense the depth of objects and to track a tiny targets trajectory. The relative error in the three-dimensional positioning is less than 7% for objects within 1.1 to 2.8 m.

Accurate three-dimensional positioning of particles is a critical task in microscopic particle research, with one of the main challenges being the measurement of particle depths. In this paper, we propose a method for detecting particle depths from their blurred images using the depth-from-defocus technique and a deep neural network-based object detection framework called you-only-look-once. Our method provides simultaneous lateral position information for the particles and has been tested and evaluated on various samples, including synthetic particles, polystyrene particles, blood cells, and plankton, even in a noise-filled environment. We achieved autofocus for target particles in different depths using generative adversarial networks, obtaining clear-focused images. Our algorithm can process a single multi-target image in 0.008 s, allowing real-time application. Our proposed method provides new opportunities for particle field research.

Human visual system relies on both monocular focusness cues and binocular stereo cues to gain effective 3D perception. Correspondingly, depth from focus/defocus (DfF/DfD) and stereo matching are two most studied passive depth sensing schemes, which are traditionally solved in separate tracks. However, the two techniques are essentially complementary: the monocular cue from DfF/DfD can robustly handle repetitive textures and occlusion that are problematic for stereo matching whereas the binocular cue from stereo matching is insensitive to defocus blurs and can resolve large depth range. In this paper, we emulate human perception and present unified learning-based techniques to conduct hybrid DfF/DfD and stereo matching. We first construct a comprehensive focal stack dataset synthesized by depth-guided light field rendering. Next, we propose different network architectures to suit various inputs, including focal stack, stereo image pair, binocular focal stack, a focus-defocus image pair and defocus-stereo image triplet. We also exploit different connection methods between the separate networks for integrating them into an optimized solution to produce high fidelity disparity maps. For experiment, we further explore different hardware setup to capture both monocular and binocular depth cues. Results show that our new learning-based hybrid techniques can significantly improve accuracy and robustness in depth estimation. [ABSTRACT FROM AUTHOR]

This paper describes how the maximum blur radius affects the depth results by depth from the defocus (DFD) method based on liquid crystal (LC) lens. Boundary frequency is determined by the maximum blur radius. It is found that if the maximum blur radius used in the calculation is larger than the real value, the depth resolution obtained is reduced; on the other hand, if one smaller than the real value is used, the depth resolution in the middle range of the scene is increased, but errors occur in the near and far planes. Using the maximum blur radius close to the real one results in the best depth results. [ABSTRACT FROM AUTHOR]

Accurate and real-time 3D burden surface topography of a blast furnace plays an important role in improving the gas flow distribution and optimizing the charging operation, and therefore has great potential to reduce both its CO2 emissions and energy consumption. However, because of the high-temperature, dusty and dark environment inside the furnace, as well as limitations in terms of installation position and viewing angle of the image capturing equipment, it is difficult to obtain clear, continuous and complete images of the burden surface and reconstruct accurate 3D topography in real time. To overcome these limitations, a novel 3D topography measurement and completion method of the burden surface is proposed in this study. First, a high-temperature industrial endoscope is designed and installed to capture the burden surface images during the blast furnace ironmaking process. Then, the details of the burden surface images are enhanced using image sharpening methods and the regions of interest of the images are extracted based on texture energy features. Next, a method based on the depth from defocus is used to calculate the depth of burden surface. Finally, considering that most of the images are incomplete because of the limitations above, a new 3D completion model is established to complete the burden surface topography based on a variational auto encoder and a generative adversarial network. On-site experiments and their validation demonstrate that the proposed method can obtain complete and accurate 3D burden surface topography and provide highly reliable topography data to optimize blast furnace operations. [ABSTRACT FROM AUTHOR]

For distance detection, a tunable aperture using liquid crystal is proposed, achieving high resolution images in real-time. The tunable aperture inserted into a camera has been designed to two sizes, which can create different Depth of Field images. By analyzing these two images, distance to object can be extracted. Initially, the aperture has large size with zero voltage. When the driving voltage is applied, the aperture transfers itself to smaller size by realignment of liquid crystal molecules in the device. The diameter of aperture is 4.4 mm with f/1.8 and 2.0 mm with f/4.0. The proposed device has low driving voltage of 2.8 V and fast response time of 9.84 ms. Compact size aperture of $10\times {10}\times 1.8$ mm3 is assembled in a camera with focal length of 8 mm which contains 5 glass lenses and 1/2.7- inch full high definition (FHD) image sensor. The tunable aperture has high stability due to no mechanically moving parts. This distance sensor can be applied to the various field of depth map application which is the self-driving car, drones and manufacturing machine. [ABSTRACT FROM AUTHOR]

Depth from defocus is a ranging method that provides depth estimation at every pixel in an image. It uses a pair of defocused images from a conventional monocular camera. To enable video-rate processing, which is important for industrial applications, an established algorithm was implemented on a field programmable gate array (FPGA). A bifurcating pipeline of 2-D filters provided the depth calculation. Multiplier and SRAM facilities on the FPGA were utilised efficiently by exploiting the filter symmetry. The coefficient and data bit-widths were limited to improve efficiency. The results compared favourably with the full width calculation. Range images were processed within 14 ms. [ABSTRACT FROM AUTHOR]

A simple and compact Depth-From-Defocus (DFD) setup, using telecentric illumination and liquid-lens based camera observation, was shown to perform well for 3D shape acquisition over extended measuring range. A further step to ameliorate the system performance is described in this paper. We focused on finding an algorithm to speed up the calibration step of the method, that automatically determines the minimum number of focal lengths to be used in the calibration and measurement procedure. As a result, the calibration is significantly shortened (up to 80% with respect to the original procedure), and the need to manually (and to some extent arbitrarily) select the focal length pairs is overcome. Measurement errors down to 0.73 mm over the measurement depth range of 130 mm, corresponding to 0.55% of the depth range are achieved, in total agreement with the original system. [ABSTRACT FROM AUTHOR]

In this work, we propose novel methods for the generation of multiple depth layers and all-in-focus images. Multiple depth layers are produced by applying a sequence of blurring and deblurring operations to determine the depth. The proposed method makes no assumptions with regard to the properties of the scene in resolving depth ambiguity in complex images. We also developed a method by which to generate all-in-focus images, in which the sharpness is preserved and instabilities imposed by conventional deblurring operations largely eliminated. This method involves updating outliers, for which depth was over-estimated, while compensating for errors in the estimation of depth imposed by deblurring. [ABSTRACT FROM AUTHOR]

We propose a new method, Depth from Asymmetric Defocus using a Color-Filtered Aperture inserted into the camera lens aperture. The proposed method provides a highly accurate depth map from a single input image by combining a depth from defocus approach with color-filtered aperture photography. [ABSTRACT FROM AUTHOR]

Scene depth estimation from a single image is a challenging problem in the field of computer vision and computer graphics. We present an approach to estimate the amount of defocus blur at edges based on the Cauchy distribution model for the DSF (points spread function). The input image was re-blurred twice respectively using two Cauchy distribution kernels, and the amount of defocus blur at the edges was obtained by the two scale parameters and the ratio between the gradients of the two re-blurred images. By propagating the blur amount at edges to the entire image, a full depth map was obtained. Experimental results on several real images demonstrate the effectiveness of our method in providing reliable depth estimation. Our method is robust to image noise, inaccurate edge location, and interference from neighboring edges, and its performance is better than existing methods based on a Gaussian DSF model. The results verify that a non-Gaussian model for DSF is feasible. [ABSTRACT FROM AUTHOR]

A digital camera industry has been developed in a decade and has a lot of applications equipped with PC and mobile phone etc. However, optics and basic principle of image projection is not changed in latest digital camera. Computational imaging is a new research area for developing new imaging strategy for combining optics and processing of the digital imaging. Camera optics, such as lens and aperture, make an image on a sensor. The quality of the image is decided by the optical transfer function called point spread function (PSF). Computation imaging engineers the PSF for imaging and produces improved quality or function of the image after processing. I will introduce some examples of the PSF engineering methods; coded aperture, focus sweep in computation imaging, and show the coding improve the depth estimation by Depth from Defocus (DFD) application. [ABSTRACT FROM AUTHOR]

Presented is an active no moving parts Depth From Defocus (DFD) system for autofocus and three-dimensional (3-D) imaging applications. The proposed system uses highly coherent laser light and its beam conditioning optics to realize an active illumination system suited for long range and large field of view imaging, overcoming the limited field of view in previous active DFD systems. This is the first time an active DFD system is implemented and experimentally demonstrated using an Electronically Controlled Variable Focus Lens (ECVFL) as an imaging lens that electrically varies the focal length to acquire images with different degrees of defocus, necessary for DFD operations. An experimental demonstration of the proposed system acting as a distance sensor is compared to an incoherent illumination projector DFD system tested over a 1 m range. Apart from showing the effectiveness of the ECVFL as an imaging optic for cameras implementing DFD based autofocus algorithms, presented experimental results for the novel system indicate a depth measurement percentage error of <11% versus a maximum 40% error obtained using an incoherent illumination source. Applications for the proposed system include imaging scenarios where high performance no moving parts autofocus is desired in weak textured scenarios, specifically for smartphone and tablet cameras where the proposed system can be miniaturized. [ABSTRACT FROM AUTHOR]

We present a new depth from defocus method based on the assumption that a per pixel blur estimate (related with the circle of confusion), while ambiguous for a single image, behaves in a consistent way when applied over a focal stack of two or more images. This allows us to fit a simple analytical description of the circle of confusion to the different per pixel measures to obtain approximate depth values up to a scale. Our results are comparable to previous work while offering a faster and flexible pipeline. [ABSTRACT FROM AUTHOR]

Photos compress 3D visual data to 2D. However, it is still possible to infer depth information even without sophisticated object learning. We propose a solution based on small-scale defocus blur inherent in optical lens and tackle the estimation problem by proposing a non-parametric matching scheme for natural images. It incorporates a matching prior with our newly constructed edgelet dataset using a non-local scheme, and includes semantic depth order cues for physically based inference. Several applications are enabled on natural images, including geometry based rendering and editing. [ABSTRACT FROM AUTHOR]

Objective This study aims to address the challenging problem of recovering the 3D depth of a scene from a single image. Most current approaches for depth recovery from a single defocused image model the point spread function as a 2D Gaussian function. However, these methods are influenced by noise, and a high quality of recovery is difficult to achieve. Method Unlike the previous depth calculations from defocus methods, we propose an approach to estimate the amount of spatially varying defocus blurs at the locations of image edges on the basis of a Cauchy distribution point-spread function model. The input defocused image is reblurred twice with two respective Cauchy distribution kernels. The amount of defocus blur at edge locations can be obtained from the ratio between the gradients of the two re-blurred images and the two scale parameters of Cauchy distribution. A full depth map is recovered by propagating the blur amount at edge locations to the entire image via matting interpolation. Result The original "Lenna" image and a rotated noise "Lenna" image are used, and a Gaussian noise is used to simulate the image noise and edge position error. Then, the average error of the Cauchy gradient ratio is compared with that of a Gaussian gradient ratio. Various real-scene image data are also used to compare our depth recovery results with those of existing methods. Experimental results show that the average error of the Cauchy gradient ratio is less than that of a Gaussian gradient ratio. Experimental results on several real images demonstrate the effectiveness of our method in estimating the defocus map from a single defocused image. Conclusion Our method is robust to image noise, inaccurate edge location, and interference of neighboring edges. The proposed method can generate more accurate scene depth maps as compared with most existing methods based on a Gaussian model. Our results also demonstrate that a non-Gaussian model for DSF is feasible. [ABSTRACT FROM AUTHOR]

Depth from defocus (DFD) is a technique that restores scene depth based on the amount of defocus blur in the images. DFD usually captures two differently focused images, one near-focused and the other far-focused, and calculates the size of the defocus blur in these images. However, DFD using a regular circular aperture is not sensitive to depth, since the point spread function (PSF) is symmetric and only the radius changes with the depth. In recent years, the coded aperture technique, which uses a special pattern for the aperture to engineer the PSF, has been used to improve the accuracy of DFD estimation. The technique is often used to restore an all-in-focus image and estimate depth in DFD applications. Use of a coded aperture has a disadvantage in terms of image deblurring, since deblurring requires a higher signal-to-noise ratio (SNR) of the captured images. The aperture attenuates incoming light in controlling the PSF and, as a result, decreases the input image SNR. In this paper, we propose a new computational imaging approach for DFD estimation using focus changes during image integration to engineer the PSF. We capture input images with a higher SNR since we can control the PSF with a wide aperture setting unlike with a coded aperture. We confirm the effectiveness of the method through experimental comparisons with conventional DFD and the coded aperture approach. [ABSTRACT FROM AUTHOR]

The LumiConSense sensor employs a thin luminescent-concentrator film, which allows lensless multifocal imaging and depth estimation at interactive rates. [ABSTRACT FROM PUBLISHER]

Three-dimensional movie acquisition and corresponding depth data is commonly generated from multiple cameras and multiple views. This technology has high cost and large size which are limitations for current consumer products, such as small camcorders and cell phone movie cameras. This research result shows that a single imager, equipped with a fast-focus microfluidic lens, produces a highly accurate depth map. On test material, the depth is found to be an average Root Mean Squared Error (RMSE) of 4.67 gray level steps (1.8%) accuracy compared to ranging data. The depth is inferred using a new Extended Depth from Defocus (EDfD), and defocus is achieved at movie speeds with a microfluidic lens. In addition, early indications show that real-time HDTV 3D movie frame rates are feasible. [ABSTRACT FROM PUBLISHER]

Highlights: [•] Formulation of an aperture shape evaluation metric. [•] Application of the metric to several depth estimation setups. [•] Evaluation of the optimized aperture shapes with a state-of-the-art algorithm. [ABSTRACT FROM AUTHOR]

• 3D positioning of the particle field is completed with high efficiency by combining EfficientNet and the DfD method. • The proposed method realizes the quick and dynamic 3D tracking of the planktons. • Cycle-GAN is introduced to further enhance the efficiency by generating simulative dataset. Our paper presents a particle field positioning method based on a developed convolutional neural network (CNN) architecture named EfficientNet, and the depth-from-defocus method which detects the depth of a particle from the blur images. The input two-dimensional images can be classified by the amount of defocus based on their characteristics. EfficientNet is then applied to estimate the depth positions of the particles. Combined with their lateral positions obtained by the traditional methods, the proposed method can determine each particle's three-dimensional (3D) position in the field. Verifiable experimental results support that our approach successfully measures the identical particle field. It is also proved to be suitable for the 3D tracking of the particles by investigating the planktons. In addition, we employed a cycle generative adversarial network to produce simulative data to reduce the workload of training data collection. Finally, the high efficiency of the proposed method shows the application potential in various fields, such as the 3D positioning, fluid investigation, and tracking of the dynamic samples. [ABSTRACT FROM AUTHOR]

A novel method is proposed for defocus map estimation. It is based on the defocus origin that is essentially the reverse of depth from defocus (DFD). The main relations among image defocus, sensor defocus, and scene defocus for an imaging system are introduced. A defocus map is deduced from the depth map and the depth map is derived from the disparity map. The full disparity map can be reconstructed using an image-matching method and our clustering segmentation algorithm. Experimental results for an interior scene and an outdoor scene demonstrate that our method is effective in defocus measurement. [ABSTRACT FROM PUBLISHER]