Search

Fluorescence lifetime imaging microscopy (FLIM) is a powerful imaging technique that enables the visualization of biological samples at the molecular level by measuring the fluorescence decay rate of fluorescent probes. This provides critical information about molecular interactions, environmental changes, and localization within biological systems. However, creating high-resolution lifetime maps using conventional FLIM systems can be challenging, as it often requires extensive scanning that can significantly lengthen acquisition times. This issue is further compounded in three-dimensional (3D) imaging because it demands additional scanning along the depth axis. To tackle this challenge, we developed a computational imaging technique called light-field tomographic FLIM (LIFT-FLIM). Our approach allows for the acquisition of volumetric fluorescence lifetime images in a highly data-efficient manner, significantly reducing the number of scanning steps required compared to conventional point-scanning or line-scanning FLIM imagers. Moreover, LIFT-FLIM enables the measurement of high-dimensional data using low-dimensional detectors, which are typically low cost and feature a higher temporal bandwidth. We demonstrated LIFT-FLIM using a linear single-photon avalanche diode array on various biological systems, showcasing unparalleled single-photon detection sensitivity. Additionally, we expanded the functionality of our method to spectral FLIM and demonstrated its application in high-content multiplexed imaging of lung organoids. LIFT-FLIM has the potential to open up broad avenues in both basic and translational biomedical research. [ABSTRACT FROM AUTHOR]

Light field cameras can obtain 3D spatial information about objects by acquiring a single image, demonstrating that light field imaging technology does not require complex light path arrangements. Owing to this capability, light field cameras have recently attracted attention from various fields. Besides, particle streak velocimetry (PSV) is a non-contact image-based flow field measurement technique widely used in optical and image processing technology. It has obvious advantages in measuring high-speed flow fields. Hence, this paper combines light field imaging technology with PSV technology and introduces a new 3D velocity measurement system that exploits the advantages of both technologies to measure flow fields with narrow windows, fast flow rates, and difficult-to-arrange measurement equipment. The system developed constructs a threedimensional flow field measurement method based on light field PSV, which is unique as no relevant research on the PSV 3D velocity measurement utilizes light field imaging. Specifically, the motion streak of a 5 μm point light source moving at different speeds was captured using a light field camera, and its 3D velocity field was verified and calculated using a computing system. The results demonstrated that the 3D flow field measurement method based on light field PSV can successfully measure 3D flow fields. [ABSTRACT FROM AUTHOR]

Deep learning is progressively emerging as a vital tool for image reconstruction in light field microscopy. The present review provides a comprehensive examination of the latest advancements in light field image reconstruction techniques based on deep learning algorithms. First, the review briefly introduced the concept of light field and deep learning techniques. Following that, the application of deep learning in light field image reconstruction was discussed. Subsequently, we classified deep learning-based light field microscopy reconstruction algorithms into three types based on the contribution of deep learning, including fully deep learning-based method, deep learning enhanced raw light field image with numerical inversion volumetric reconstruction, and numerical inversion volumetric reconstruction with deep learning enhanced resolution, and comprehensively analyzed the features of each approach. Finally, we discussed several challenges, including deep neural approaches for increasing the accuracy of light field microscopy to predict temporal information, methods for obtaining light field training data, strategies for data enhancement using existing data, and the interpretability of deep neural networks.

Hyperspectral imaging is an emerging technology that has gained significant attention in the medical field due to its ability to provide precise and accurate imaging of biological tissues. The current methods of hyperspectral imaging, such as filter-wheel, snapshot, line-scanning, and push-broom cameras have limitations such as low spatial and spectral resolution, slow acquisition time. New developments on the field of light field cameras show the potential to overcome these limitations. In this paper, we use a novel hyperspectral lightfield camera and try to combine the capability of hyperspectral and 3D analysis. For this purpose we calibrate our system and test it during two ENT-surgeries to show its potential for improving surgical outcomes. The micro-lenses of the camera map 66 spectral sub-images onto the sensor allowing to reconstruct the spectral behavior of the captured scene in the spectral range of 350-1000nm. In addition, we use the sensor data to apply a 3D camera calibration pipeline to allow 3D surface reconstruction. We captured 26 calibration images and achieved calibration results in accordance to stated company data. The best calibration showed a re-projection error of 0.55 px. Further, we tested the camera during a parotidectomy and a neck-dissection. The extracted reflectance spectra of the selected venal and arterial regions correspond perfectly to the spectrum of oxygenated and deoxygenated hemoglobin. For the first time, up to our knowledge, a hyperspectral lightfield camera has been used during a surgery. We were able to continuously capture images and analyze the reconstructed spectra of specific tissue types. Further, we are able to use the sensor data of the micro-lens projections to calibrate the multilens camera system for later intraoperative measurement tasks.

Light field tomography, an optical combustion diagnostic technology, has recently attracted extensive attention due to its easy implementation and non-intrusion. However, the conventional iterative methods are high data throughput, low efficiency and time-consuming, and the existing machine learning models use the radiation spectrum information of the flame to realize the parameter field measurement at the current time. It is still an offline measurement and cannot realize the online prediction of the instantaneous structure of the actual turbulent combustion field. In this work, a novel online prediction model of flame temperature instantaneous structure based on deep convolutional neural network and long short-term memory (CNN-LSTM) is proposed. The method uses the characteristics of local perception, shared weight, and pooling of CNN to extract the three-dimensional (3D) features of flame temperature and outgoing radiation images. Moreover, the LSTM is used to comprehensively utilize the ten historical time series information of high dynamic combustion flame to accurately predict 3D temperature at three future moments. A chaotic time-series dataset based on the flame radiation forward model is built to train and validate the performance of the proposed CNN-LSTM model. It is proven that the CNN-LSTM prediction model can successfully learn the evolution pattern of combustion flame and make accurate predictions. [ABSTRACT FROM AUTHOR]

Robust lane detection is imperative for the realization of intelligent transportation. Recently, vision-based systems that employ deep convolution neural networks (CNNs) for lane detection have made considerable progress. However, for better generalization under various road conditions learning-based methods require excessive training data, which becomes non-trivial in challenging conditions such as illumination variation, shadows, false lane lines, and worn lane markings, etc. In this paper, we propose a light field (LF) based lane detection method that utilizes the additional angular information for improved prediction and increased robustness. Two different LF representations are investigated to study the possibility of maximum performance improvement and minimal additional computation cost and data labeling efforts. Experimental results successfully demonstrate that the proposed approach improves the prediction of the lane line point coordinates and is significantly robust against the aforementioned adverse conditions.

Deep learning (DL) provides an effective approach for light field (LF) reconstruction that aims to synthesize novel views from sparsely-sampled views. However, it is challenging to address domain asymmetry when adopting spatial-angular interaction LF reconstruction methods. To overcome this problem, a view-selective angular feature extraction block (VS-LFAFE) is proposed to obtain full-resolution angular features that enumerate whole viewpoints in a macropixel. By applying the VS-LFAFE, a novel LF reconstruction method is proposed, consisting of two subblocks: a spatial-angular feature extraction and fusion block, and an angular upsampling block. Experimental results demonstrate the effectiveness of the VS-LFAFE, and validate that the proposed method can achieve superior performance compared with the state-of-the-art methods.

Depth estimation for light field (LF) images is the cornerstone of many applications of light field cameras, such as 3D reconstruction, defects inspection, face liveness detection, and so forth. In recent years, convolutional neural network (CNN) has dominated the primary workhorse for depth estimation. However, the interpretability of the network and the accuracy of the depth estimation results still need to be improved. This paper uses the conditional random field (CRF) theory to explain and model the LF depth estimation. Further, from the perspective of sequence analysis, we extract the sequence features of epipolar plane image (EPI) patches with recurrent neural network (RNN) and serve as the unary term of the energy function in the CRF. Then, a unified neural network (called as LFRNN) is designed to solve the CRF and get the disparity map. Our LFRNN builds upon two-stage architecture, involving a local depth estimation and a depth refinement. In the first part, we design an RNN to analyze the vector sequences in EPI patches and obtain local disparity values. There are two thinking behind the design of this part. The first is the general principle that the slope of the straight line in the EPI is inversely proportional to the depth; the second is our unique observation that those straight lines are distributed in vector sequences. In the second part, continuous CRF is used to optimize the output of the first part. We train LFRNN on a synthetic LF dataset and test it on both synthetic and real-world LF datasets. Quantitative and qualitative results validate the superior performance of our LFRNN over the state-of-the-art methods.

In order to accurately recover the complete shape of the measured object with partial occlusion, a system for light field 3-D imaging based on multi-line structured light marking was proposed and a structured light field 3-D imaging method based on the Hough transform was proposed for the image processing problem in this system. The extracted sub-pixel fringe center coordinates were used to Hough transform, then the voting distribution in the Hough parameter space was analyzed, and adaptive range voting and adaptive window strategies were designed. Consequently, the parameters of multiple straight lines in the epipolar plane image can be accurately determined by the proposed method, which does not need to encode the fringe level. The experimental results show that the mean deviation and standard deviation of the fitted plane are 0.0096 mm and 0.0074 mm respectively. The complete shape of measured object is accurately obtained, which utilities the occlusion of different sub-aperture images is inconsistent in light field imaging. The result is helpful for solving the occlusion problem in the 3-D measurement process, and this work provides a reference for the measurement method to obtain complete and efficient 3-D data. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Many applications of light field (LF) imaging have been limited by the spatial-angular resolution problem, hence the need for efficient super-resolution techniques. Recently, learning-based solutions have achieved remarkably better performances than traditional super-resolution (SR) techniques. Unfortunately, the learning or training process relies heavily on the training dataset, which could be limited for most LF imaging applications. In this paper, we propose a novel LF spatial-angular SR algorithm based on zero-shot learning. We suggest learning cross-scale reusable features in the epipolar plane image (EPI) space, and avoiding explicitly modeling scene priors or implicitly learning that from a large number of LFs. Most importantly, without using any external LFs, the proposed algorithm can simultaneously super-resolve a LF in both spatial and angular domains. Moreover, the proposed solution is free of depth or disparity estimation, which is usually employed by existing LF spatial and angular SR. By using a simple 8-layers fully convolutional network, we show that the proposed algorithm can generate comparable results to the state-of-the-art spatial SR. Our algorithm outperforms the existing methods in terms of angular SR on multiple groups of public LF datasets. The experiment results indicate that the cross-scale features can be well learned and be reused for LF SR in the EPI space. [ABSTRACT FROM AUTHOR]

The luminance dynamic range in natural scenes is exceedingly wide. However, capturing the wide dynamic range information of natural scenes through a single exposure is challenging for commercial light field cameras. One solution to expand the luminance dynamic range of captured light field images is to employ multi-exposure imaging techniques. Nevertheless, this approach may introduce various distortions during the image fusion procedure, especially for dynamic scenes. To evaluate the visual quality of multi-exposure fused light field images (MEFLFIs) and compare the performance of different multi-exposure light field fusion methods, we initially conduct a subjective quality assessment on MEFLFIs for dynamic scenes and establish the first MEFLFI benchmark dataset, which comprises 480 fused light field images generated by eleven state-of-the-art multi-exposure fusion algorithms and two tone-mapping algorithms, along with their corresponding subjective scores. Subsequently, a novel specially designed objective quality assessment metric for MEFLFIs is developed, which incorporates both local–global joint features and angular quality-aware features. The key contribution of this work lies in establishing a pioneering benchmark dataset for MEFLFIs along with a novel tailored objective quality metric. Experimental results conducted over our established MEFLFI database demonstrate that the proposed metric achieves superior performance, with Pearson linear correlation coefficient (PLCC) and Spearman rank correlation coefficient (SROCC) values of 0.8895 and 0.8658 respectively, representing a significant improvement of approximately 5.7 % and 4.9 % compared to the second-best specifically-designed LFI quality metric available. Furthermore, ablation experiments have confirmed that the integration of local–global joint features and angular features significantly enhances the performance of our objective metric, resulting in a better alignment with human visual perception. [ABSTRACT FROM AUTHOR]

Fast neutron identification and spectroscopy is of great interest to nuclear physics experiments. Using the neutron elastic scattering, the fast neutron momentum can be measured. Wang and Morris introduced the theoretical concept that the initial fast neutron momentum can be derived from up to three consecutive elastic collisions between the neutron and the target, including the information of two consecutive recoil ion tracks and the vertex position of the third collision or two consecutive elastic collisions with the timing information. Here, we also include the additional possibility of measuring the deposited energies from the recoil ions. In this paper, we simulate the neutron elastic scattering using the Monte Carlo N-Particle Transport Code (MCNP) and study the corresponding neutron detection and tracking efficiency. The corresponding efficiency and the scattering distances are simulated with different target materials, especially natural silicon (92.23%28Si, 4.67%29Si, and 3.1%30Si) and helium-4 (4He). The timing of collision and the recoil ion energy are also investigated, which are important characters for the detector design. We also calculate the ion traveling range for different energies using the software, “The Stopping and Range of Ions in Matter (SRIM)”, showing that the ion track can be most conveniently observed in 4He unless sub-micron spatial resolution can be obtained in silicon.

Deep learning is progressively emerging as a vital tool for image reconstruction in light field microscopy. The present review provides a comprehensive examination of the latest advancements in light field image reconstruction techniques based on deep learning algorithms. First, the review briefly introduced the concept of light field and deep learning techniques. Following that, the application of deep learning in light field image reconstruction was discussed. Subsequently, we classified deep learning-based light field microscopy reconstruction algorithms into three types based on the contribution of deep learning, including fully deep learning-based method, deep learning enhanced raw light field image with numerical inversion volumetric reconstruction, and numerical inversion volumetric reconstruction with deep learning enhanced resolution, and comprehensively analyzed the features of each approach. Finally, we discussed several challenges, including deep neural approaches for increasing the accuracy of light field microscopy to predict temporal information, methods for obtaining light field training data, strategies for data enhancement using existing data, and the interpretability of deep neural networks., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2024 Lin, Tian, Zhang, Zhu and Wang.)

Compressed sensing using color-coded masks has been recently considered for capturing light fields using a small number of measurements. Such an acquisition scheme is very practical, since any consumer-level camera can be turned into a light field acquisition camera by simply adding a coded mask in front of the sensor. We present an efficient and mathematically grounded deep learning model to reconstruct a light field from a set of measurements obtained using a color-coded mask and a color filter array (CFA). Following the promising trend of unrolling optimization algorithms with learned priors, we formulate our task of light field reconstruction as an inverse problem and derive a principled deep network architecture from this formulation. We also introduce a closed-form extraction of information from the acquisition, while similar methods found in the recent literature systematically use an approximation. Compared to similar deep learning methods, we show that our approach allows for a better reconstruction quality. We further show that our approach is robust to noise using realistic simulations of the sensing acquisition process.

To study the three-dimensional motion characteristics of bubbles in a bubble slurry bed, a simultaneous measurement method for bubble size and velocity based on a light field imaging particle tracking velocimetry (PTV) technology was proposed. A light-field PTV bubble measurement system was constructed, the light-field camera was used to collect two consecutive light field images. After refocusing calculation, image processing and sharpness evaluation, the size and three-dimensional coordinates of the bubbles were calculated combined with the calibration results. The PTV technology was used to calculate the three-dimensional velocity of the bubble, and the simultaneous measurement of bubble size and three-dimensional velocity was completed. The evaluation of the light field PTV system and the measurement of bubble size and three-dimensional velocity were carried out. The results show that the relative error of the diameter measurement is less than 1.60%, the repeatability error of the diameter measurement is less than 2. 48%, and the relative error of the depth position measurement is less than 7.14% . When the air flow increases from 0.1 L/min to 0.5 L/min, the average equivalent diameter of the bubbles increases from 3. 85 mm to 4. 36 mm, and the average bubble velocity increases from 0. 34 m/s to 0. 38 m/s. The light field imaging PTV technology can be used for simultaneous measurement of the bubble size and the three-dimensional velocity. [ABSTRACT FROM AUTHOR]

An unfocused plenoptic light field (LF) camera places an array of microlenses in front of an image sensor in order to separately capture different directional rays arriving at an image pixel. Using a conventional Bayer pattern, data captured at each pixel is a single color component (R, G or B). The sensed data then undergoes demosaicking (interpolation of RGB components per pixel) and conversion to an array of sub-aperture images (SAIs). In this paper, we propose a new LF image coding scheme based on graph lifting transform (GLT), where the acquired sensor data are coded in the original captured form without pre-processing. Specifically, we directly map raw sensed color data to the SAIs, resulting in sparsely distributed color pixels on 2D grids, and perform demosaicking at the receiver after decoding. To exploit spatial correlation among the sparse pixels, we propose a novel intra-prediction scheme, where the prediction kernel is determined according to the local gradient estimated from already coded neighboring pixel blocks. We then connect the pixels by forming a graph, modeling the prediction residuals statistically as a Gaussian Markov Random Field (GMRF). The optimal edge weights are computed via a graph learning method using a set of training SAIs. The residual data is encoded via low-complexity GLT. Experiments show that at high PSNRs—important for archiving and instant storage scenarios—our method outperformed significantly a conventional light field image coding scheme with demosaicking followed by High Efficiency Video Coding (HEVC). [ABSTRACT FROM AUTHOR]

Copyright of Frontiers of Information Technology & Electronic Engineering is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Fast neutron identification and spectroscopy is of great interest to nuclear physics experiments. Using the neutron elastic scattering, the fast neutron momentum can be measured. Wang and Morris introduced the theoretical concept that the initial fast neutron momentum can be derived from up to three consecutive elastic collisions between the neutron and the target, including the information of two consecutive recoil ion tracks and the vertex position of the third collision or two consecutive elastic collisions with the timing information. Here, we also include the additional possibility of measuring the deposited energies from the recoil ions. In this paper, we simulate the neutron elastic scattering using the Monte Carlo N-Particle Transport Code (MCNP) and study the corresponding neutron detection and tracking efficiency. The corresponding efficiency and the scattering distances are simulated with different target materials, especially natural silicon (92.23% 28 Si, 4.67% 29 Si, and 3.1% 30 Si) and helium-4 ( 4 He). The timing of collision and the recoil ion energy are also investigated, which are important characters for the detector design. We also calculate the ion traveling range for different energies using the software, "The Stopping and Range of Ions in Matter (SRIM)", showing that the ion track can be most conveniently observed in 4 He unless sub-micron spatial resolution can be obtained in silicon. [ABSTRACT FROM AUTHOR]

Fluorescence lifetime imaging microscopy (FLIM) is a powerful imaging technique that enables the visualization of biological samples at the molecular level by measuring the fluorescence decay rate of fluorescent probes. This provides critical information about molecular interactions, environmental changes, and localization within biological systems. However, creating high-resolution lifetime maps using conventional FLIM systems can be challenging, as it often requires extensive scanning that can significantly lengthen acquisition times. This issue is further compounded in three-dimensional (3D) imaging because it demands additional scanning along the depth axis. To tackle this challenge, we developed a computational imaging technique called light-field tomographic FLIM (LIFT-FLIM). Our approach allows for the acquisition of volumetric fluorescence lifetime images in a highly data-efficient manner, significantly reducing the number of scanning steps required compared to conventional point-scanning or line-scanning FLIM imagers. Moreover, LIFT-FLIM enables the measurement of high-dimensional data using low-dimensional detectors, which are typically low cost and feature a higher temporal bandwidth. We demonstrated LIFT-FLIM using a linear single-photon avalanche diode array on various biological systems, showcasing unparalleled single-photon detection sensitivity. Additionally, we expanded the functionality of our method to spectral FLIM and demonstrated its application in high-content multiplexed imaging of lung organoids. LIFT-FLIM has the potential to open up broad avenues in both basic and translational biomedical research., Competing Interests: Competing interests statement:Edoardo Charbon holds the position of Chief Scientific Officer at Fastree3D, a company that specializes in the manufacturing of LiDARs for the automotive market. Claudio Bruschini and Edoardo Charbon are also cofounders of Pi Imaging Technology. Additionally, Liang Gao has a financial interest in Lift Photonics, which commercializes the LIFT technology for FLIM applications. However, it is important to note that none of these companies were involved in the research presented in this paper. The authors disclose UCLA provisional patent filing. Research support was provided by the National Institutes of Health (R01HL165318 and RF1NS128488).

• A novel approach is proposed to retrieve the 3-D soot absorption coefficient and temperature of flames. • Achieves higher accuracy in 3-D temperature reconstruction and reduces reconstruction time. • Numerical simulations are carried out to evaluate the proposed approach. • The performance of the proposed method is evaluated through ethylene/air diffusion flames. The accuracy and efficiency of 3-D flame temperature field reconstruction using the light field imaging technique heavily depend on soot radiation characteristics. In this study, we employ the line-of-sight attenuation method to reconstruct the soot absorption coefficient distribution in a pure absorbing flame. Utilizing these soot absorption coefficients, the radiative transfer equation is effectively transformed within the framework of the light field imaging technique into a linear inverse problem and also outlines the flame boundary. This proposed strategy reduces the unwanted detection rays significantly, thus eliminating the extensive computational processing. Consequently, the proposed approach substantially enhances the accuracy and efficiency of flame temperature reconstruction. Numerical simulations were carried out on a bimodal asymmetric flame to validate the noise tolerance capabilities, investigate the effects of varying voxel numbers on flame division and carry out a comparative study. Experimental studies were also conducted to reconstruct flame temperature and soot absorption coefficient distributions under different combustion operating conditions. Thermocouple measurements were performed and compared with the reconstructed temperatures. Outcomes achieved from both numerical simulations and experimental studies demonstrate the feasibility, accuracy and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

• L p -norm relaxation approach combined with non-negative constraint is proposed. • Impact of sampling characteristics on ill-posedness of inverse problem is analyzed. • Robustness and validity of L p -norm are investigated under measurement noise. • Experiment system is built to implement premixed flame 3D temperature measurement. Optical imaging technology based on flame spontaneous radiation is a promising tool for combustion diagnosis. However, the existing radiation thermometry technology faces the problems of low angular resolution and low efficiency in solving radiative transfer. Therefore, a novel optical imaging model is proposed, which combines light field imaging theory with equation-solving integral equation method based on radiation distribution factor to effectively decouple the radiative properties related to transfer process and the temperature-related source terms. Aiming at the problem that traditional method may cause over-smooth effect, L p -norm regularization based on non-negative constraint is established. Results show that L p -norm can improve reconstruction quality and robustness, but p value needs to be reasonably selected according to noise level and flame inhomogeneity. Experimental study of ethylene/air-premixed flames shows that the methodology can realize 3D measurement (900 ∼ 1550 K). Compared with laser absorption spectroscopy and thermocouple, error is better than 3 % at height above burner 15 mm. [ABSTRACT FROM AUTHOR]

• A novel learning-based method is proposed for ghost-free high dynamic range (HDR) light field imaging. • A multi-scale architecture integrating deformable alignment module and angular embedding module is designed. • A new large-scale benchmark dataset is established to serve the HDR light field imaging task for dynamic scenes. • The proposed method achieves superior spatial quality and preserves accurate angular consistency. Light field (LF) imaging is an effective way to enable immersive applications. However, limited by the potential well capacity of the image sensor, the acquired LF images suffer from low dynamic range and are thus prone to under-exposure or over-exposure. High dynamic range (HDR) LF imaging is an efficacious avenue to improve the LF imaging's dynamic range. Unfortunately, for dynamic scenes, existing methods are inclined to produce ghosting artifacts and lose details in the saturated regions, while potentially damaging the parallax structure of generated HDR LF images. To address the above challenges, in this paper, we propose a new ghost-free HDR LF imaging method based on a deformable aggregation and angular embedding network. Specifically, considering the four-dimensional geometric structure of the LF image, a deformable alignment module is first designed to handle dynamic regions in the spatial domain, and then the aligned spatial features are fully fused through an aggregation operation. Subsequently, an angular embedding module is constructed to explore angular information to enhance the aggregated spatial features. Based on this, the above two modules are cascaded in a multi-scale manner to achieve multi-level feature extraction and enhance the feature representation ability. Finally, a decoder is leveraged to recover the ghost-free HDR LF image from the enhanced multi-scale features. For performance evaluation, this paper establishes a large-scale benchmark dataset with multi-exposure inputs and ground truth images. Extensive experimental results show that the proposed method generates visually pleasing HDR LF images while preserving accurate angular consistency. Moreover, the proposed method surpasses the state-of-the-art methods in both quantitative and qualitative comparisons. The code and dataset will be available at https://github.com/YeyaoChen/HDRLFI. [ABSTRACT FROM AUTHOR]

• Light field imaging considers flame radiation, wall reflection and occlusion in confined combustor. • Reflection characteristics and radiative transfer mechanism of the chamber wall are analyzed. • Region segmentation and regularization are introduced into flame tomography within wider temperatures. • The proposed novel method has good accuracy, fluctuation suppression and anti-noise ability in 3D imaging. Optical tomography has been demonstrated to be a powerful tool for the three-dimensional (3D) measurement of the dynamic characteristics of turbulent flame. Many practical visualizations of reaction flows require imaging through optical windows on the combustor, which have complex interactions with wall temperature, reflection characteristics, and limited optical access, which may seriously influence the imaging quality. Previous work focused on unconfined flame imaging, and lacked the study of the multiple reflection effects of the wall and the tomographic reconstruction within a wide temperature range. To solve these problems, the effect of chamber wall radiation on the photothermal information is analyzed in the forward problem, and the flame temperature with wall radiation is reconstructed in the inverse problem. In the forward problem, the light field acquisition model of spontaneous emission of the confined flame is developed through the backward Monte Carlo method based on the radiation distribution factor, which can comprehensively consider the essential effects such as radiative attenuation, wall reflection, medium scattering interference, and limited optical access. The recorded light field signals indicate that the reflection pattern and transfer mechanism of the wall material could affect the contribution of radiative source terms in different discrete domains to the detected energy. A novel tomography algorithm, namely, the adaptive threshold segmentation iterative regularization (ATSIR) method, has been proposed to extend the temperature range of tomography. The key of this method is to carry out a two-step reconstruction of the high and low-temperature regions through the temperature threshold, and introduce the prior smoothing information to alleviate the ill-posedness of tomography. The visualization results show that the well-established algorithms (e.g., least-square QR-factorization and regularization) have lower reconstruction quality for the combustion field within a wide temperature range, even if the regularization term is added. However, the proposed ATSIR method has the advantages of better suppression of fluctuations, smaller reconstruction error, and stronger anti-noise ability. The detailed analysis of the radiative transfer mode and 3D temperature visualization of the combustion field in a confined space can provide a valuable guideline for the safety design of the engine combustion chamber and fuel utilization. [ABSTRACT FROM AUTHOR]

2D optical imaging systems cause irreparable stereo visual feature loss in the pixel-level semantic segmentation task, especially in challenging scenes. To overcome this limitation, light field imaging with microlens arrays can acquire the 4D multi angle subaperture images, providing a viable alternative to semantic segmentation. However, these images are redundant and difficult to process. Therefore, we propose a macro generalized epipolar-plane image (macro-GEPI) representation. The macro-GEPI is obtained by slicing the subaperture images, stacking the slice images in sequence, and finally aligning them. This new image representation can clearly highlight the spatial-angular interactive information of different objects in one image. Based on this, we propose a convolutional neural network (CNN) that can independently learn the light field spatial-angular information for semantic segmentation. We also present a light field dataset composed of 1840 macro-GEPIs with annotations. We have studied the proposed network and compared it to state-of-the-art (SOTA) algorithms. The experiments showed that our network outperformed SOTA algorithms on the macro-GEPI dataset and a benchmark light field dataset, especially in challenging scenes. Moreover, our algorithm achieved a high mean intersection over union of 90.83%, demonstrating the reliability of the light field macro-GEPI representation and the effectiveness of our CNN algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a method based on polarization characteristics of the light field is proposed to remove specular highlight and accurately estimate the depth information for highly reflective surfaces. This method innovatively combines the polarization characteristics and the advantages of the light field imaging to effectively suppress the specular highlight and realize the visual imaging as well as depth estimation. Firstly, a focused light field camera which spatial and angular resolutions can be flexibly adjusted as needed is designed and used to capture the light field images of the target scenes, and the image correction algorithm is performed on the target images to correct image distortion. Then, the relationship between the image grayscale and the polarization angle is established to remove the specular highlight according to the Stokes parameters of light field, and the color transfer algorithm is also used to restore output images. Finally, a method based on entropy mean gradient ratio is designed to quantitatively evaluate the effectiveness of our method. In addition, a depth estimation system of focused light field is built independently to reconstruct the depth information for different highly reflective surfaces by using the proposed method. The experimental results show that our proposed method is suitable for local specular highlight removal under diffuse light source, which provides a new imaging method for highly reflective surfaces. • A method based on polarization of light field is proposed to remove specular highlight and estimate the depth information. • A focused light field camera which spatial and angular resolutions can be flexibly adjusted as needed is designed. • A method based on entropy mean gradient ratio is designed to quantitatively evaluate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

The optical illusion affects depth-sensing due to the limited and specific light-field information acquired by single-lens imaging. The incomplete depth information or visual deception would cause cognitive errors. To resolve this problem, an intelligent and compact depth-sensing meta-device that is miniaturized, integrated, and applicable for diverse scenes in all light levels is demonstrated. The compact and multifunction stereo vision system adopts an array with 3600 achromatic meta-lenses and a size of 1.2 × 1.2 mm 2 to measure the depth over a 30 cm range with deep-learning support. The meta-lens array can act as multiple imaging lenses to collect light field information. It can also work with a light source as an active optical device to project a structured light. The meta-lens array can serve as the core functional component of a light-field imaging system under bright conditions or a structured-light projection system in the dark. The depth information in both ways can be analyzed and extracted by the convolutional neural network. This work provides a new avenue for the applications such as autonomous driving, machine vision, human-computer interaction, augmented reality, biometric identification, etc., (© 2022 Wiley-VCH GmbH.)

• Reconstruction method of temperature and soot volume fraction is proposed. • The proposed method benefits from WNNLS and light field imaging techniques. • WNNLS algorithm improves the equality of radiative signals in reconstruction. • The proposed method has a good anti-nosie ability. • Experimental studies demonstate the effectiveness of the proposed method. The flame temperature and soot volume fraction are important parameters for combustion diagnosis, which reflect the combustion reaction process and play a significant role in controlling pollutants emissions. This research introduces a direct method based on Weighted Non-Negative Least Squares (WNNLS) and light field imaging techniques to recover the flame temperature and soot volume fraction simultaneously. This method considers the flame self-absorption effect. Numerical studies were conducted on both Non-Negative Least Squares and proposed algorithms to compare the reconstruction performance. The anti-noise ability of the proposed reconstruction method was also investigated by adding various noises to radiative intensity. The effectiveness of the proposed method was also further verified by experimental studies on bimodal ethylene-diffusion flames. Thermocouple measurements were conducted and compared with the reconstructed temperatures. Both the numerical and experimental studies manifest that the proposed direct reconstruction method combing with WNNLS and light field imaging techniques is effective and accurate. [ABSTRACT FROM AUTHOR]

• The core component of this system is a ZnO-LCMLA, which induces horizontal orientation of LC molecules via a ZnO microstructure to effectively overcome physical defects caused by orientation, such as mechanical friction. • High-resolution imaging can be achieved by combining the redundant characteristics of light field images with the proposed joint optimization algorithm based on convex optimization theory. • A series of experiments have shown that the ZnO microstructure can improve the orientation of LC molecules and the robustness of the light field imaging system via the fabricated ZnO-LCMLA. The imaging system developed is adaptable, simple, and suitable for high-resolution reconstruction. We proposed a liquid crystal (LC) microlens array (MLA) based on zinc oxide (ZnO) microstructure orientation and used it in a light field imaging system. The system uses convex optimization theory to jointly optimize the multi-voltage light field information and obtain a high-resolution imaging method. The core element in this system is an LCMLA, which induces the horizontal orientation of LC molecules by ZnO microstructure to effectively overcome the physical defects caused by orientation, such as mechanical friction. Combined with the proposed joint optimization high-resolution algorithm based on convex optimization theory, high-resolution imaging can be achieved using the redundant characteristics of light field images. The experiments show that the proposed method's PSNR of the reconstructed image is about 32 dB compared with the conventional method. The designed imaging system is flexible, simple, and suitable for high-resolution reconstruction. [Display omitted] This study demonstrated the use of a liquid crystal (LC) microlens array (MLA) based on zinc oxide (ZnO) microstructure orientation in a light field imaging system. The system uses convex optimization theory to jointly optimize the multi-voltage light field information and obtain a high-resolution imaging method. [ABSTRACT FROM AUTHOR]

• A 3D measurement method of bubbles through the use of edge gradient segmentation by a single light field camera. • A 3D reconstruction method of bubbles with orthogonal dual light field cameras. • Influence of the bubble size on the bubble depth measurement was revealed. Based on the light field imaging technology, a three-dimensional(3D) measurement method of bubbles through the use of edge gradient segmentation (TMBEGS) is proposed in this paper. From the characteristic that the bubble edge image has great gray gradient, the coordinates of the bubble edge pixels are determined. The depths of the edge pixels are further derived by evaluating the sharpness of the edge pixels in the refocused light field images. On this basis, the depth coordinate of the bubble is calculated by averaging the depths of all edge pixels, and the 3D measurement of the bubble is then achieved by combining the centroid coordinate and equivalent diameter of the bubble. Finally, experiments are carried out to evaluate TMBEGS on a calibration setup and a bubbling bed experimental device. Results indicated that the relative errors of the bubble diameter and depth by TMBEGS are less than 1.18 % and 5.93 %, which are better than 2.99 % and 17.51 % by the existing method, respectively. On the bubbling bed experimental device, the equivalent diameter of bubbles is increased by 5.69 % compared with the existing method, and the standard deviation of the bubble depth is less than 1.78 mm, which is 21.3 % lower than the existing method. With the increase of the bubble equivalent diameter, the bubble surface deformation intensifies under the effects of the bubble surface tension, viscous force, and inertia force, which affects the imaging quality of the bubble, and reduces the measurement accuracy of the bubble edge and depth. These results illustrated that TMBEGS has high measurement accuracy of the bubbles and is capable of charactering the bubbles in the gas–liquid two-phase flow system. [ABSTRACT FROM AUTHOR]

• A meshed axisymmetric flame light transmission model was put forward. • A light field imaging model was developed for the meshed flame. • The effects of non-uniform radiative properties on light field images were analyzed. • Flame temperature was reproduced from light field images by deconvolution method. • Reconstructed results are comparable at temperature among 1400 K-2000 K. Axisymmetric flames are widely used in research on flame combustion mechanisms and mass and heat transfer because of their simple structure and applicability to complex reactions and transfer calculations. A new optical diagnostic method named light field camera has been validated as applicable for online detection of the temperature field of an axisymmetric flame. Reconstruction algorithms can be used to estimate the distribution of the flame temperature from the light field images and validated by setting a known distribution of temperature and radiative properties. In the present study, a model of meshed axisymmetric flame light field imaging was put forward. The complicated distribution of temperature and radiative properties were considered in the model after validation using two absorption coefficients arranged at different region. An axisymmetric laminar coflow diffusion ethylene flame was modeled to generate input data. Then, the Lucy–Richardson and nearest neighbor filtering deconvolution algorithms were applied to the refocused light field images for sectional reconstruction of the temperature field. Under the condition of high temperature and soot generated from ethylene flame, the obtained reconstructed temperature was comparable to the input temperature from 1400 K to 2000 K, within the detectable radiative intensity range of the light field camera. [ABSTRACT FROM AUTHOR]

Light field cameras show great potential for use in retinal imaging. The ability to refocus images and remove glare post-acquisition provides great benefits over current fundus cameras. Previous light field fundus cameras did not have the spatial resolution required for retinal screening. In this work, we design a light field fundus camera based on the specifications prescribed by the ISO standard for fundus cameras. We build a camera according to the developed specifications and test it using the method prescribed by the standard. We validate the design procedure by showing that the resolutions achieved by the constructed camera were similar to those achieved by the theoretical design. We additionally show that we can achieve a spatial resolution of 33.3 lp/mm which is a significant improvement over 17.96 lp/mm obtained in previous work. Given a larger sensor and custom microlens array, the proposed design procedure could feasibly provide the spatial resolution required for diagnostic retinal imaging while keeping the preferable qualities of a light field camera. [ABSTRACT FROM AUTHOR]

Light field (LF) imaging, as an emerging temperature measurement technique, can simultaneously capture the intensity and direction information of radiation in the combustion flow field and has recently attracted extensive attention. However, the dense sampling of the LF introduces a large amount of redundant information and makes the plenoptic data susceptible to noise pollution, which can significantly affect the performance of 3D temperature field reconstruction. To address this problem, a novel approach, i.e., light field compression and noise reduction (LFCNR), is proposed to extract the main features of the projection matrix and separate the noise-related and signal-related subspaces of the plenoptic data, thus improving the reconstruction accuracy and efficiency. The performance of the proposed LFCNR method and the effect of the hyper-parameters on reconstructed quality are investigated thoroughly. It has been observed that the proposed LFCNR-based method for 3D temperature tomography imaging has higher accuracy, more efficiency, and better anti-noise ability compared to the well-established method. LFCNR can improve the reconstructed temperature accuracy by approximately 20%, and the calculation time is shortened to 10%. Priori smoothing is introduced into the optimization objective to constrain the fluctuation of the retrieval temperature, called LFCNR-PS, which can further improve the reconstructed accuracy of axisymmetric and non-axisymmetric fields about 1.79% and 1.23%. Furthermore, experiments of the LFCNR-based method are carried out to reconstruct the 3D temperature field of ethylene diffusion flames. The reconstructed temperature slices demonstrate that the flame topology can be clearly and rapidly identified at different depths. The efficiency and robustness of the proposed method can be applied to the harsh environment, which will be beneficial to the reliable operation of thermal equipment, the study of turbulent chemical reaction mechanisms, fuel utilization, and the optimization design of the engine combustion chamber. [ABSTRACT FROM AUTHOR]

In this paper, we study the spectral properties of re-parameterized light field. Following previous studies of the light field spectrum, which notably provided sampling guidelines, we focus on the two plane parameterization of the light field. However, we introduce additional flexibility by allowing the image plane to be tilted and not only parallel. A formal theoretical analysis is first presented, which shows that more flexible sampling guidelines (i.e. wider camera baselines) can be used to sample the light field when adapting the image plane orientation to the scene geometry. We then present our simulations and results to support these theoretical findings. While the work introduced in this paper is mostly theoretical, we believe these new findings open exciting avenues for more practical application of light fields, such as view synthesis or compact representation. • The spectral properties of re-parameterized light field are studied. • Additional flexibility is introduced by allowing the image plane to be tilted. • A formal theoretical analysis is presented. • It is shown wider camera baselines can be used to capture light fields. • Simulations results are presented to support these theoretical findings. [ABSTRACT FROM AUTHOR]

• An axial resolution enhancement light-sheet light-field fluorescence microscopy with defocus modulation using an ETL is presented. • Combining light field reconstruction theory and the depth estimation method based on EPI, the depth position of each scene point of the sample within the thickness of a single light sheet can be estimated. • The axial resolution is improved to less than 0.5 μ m under a 6- μ m -thick light sheet illumination. • It provides a potential for realizing both high axial resolution and large field of view in a light sheet fluorescence microscopy. Light sheet fluorescence microscopy (LSFM) is a competitive 3D fluorescence imaging method for live biological specimens owing to its simultaneously fast volumetric imaging speed, high contrast, and low phototoxicity. Usually, the axial resolution is hard to be improved in an optical microscopy, especially in a LSFM with a light sheet illuminating. Here, the axial resolution is improved by using the light field imaging that can treat the light sheet with certain thickness as a volumetric imaging but not simply two-dimensional wide-field imaging. By using the light field reconstruction with backprojection (LFBP) approach and depth estimation algorithm, the axial resolution of the modified LSFM can be improved without loss of spatial resolution. In the proof-of-principle experiment, the axial resolution is improved to less than 0.5 μ m under a 6- μ m -thick light sheet illumination. With the proposed method, it provides a potential for realizing both high axial resolution and large field of view in a LSFM. [ABSTRACT FROM AUTHOR]

To observe targets in expanded depths of view in light field imaging, we design an open-source light field parallel refocusing code-free GUI tool named OpenRefocus, which is based on classical spatial projection integration. OpenRefocus has been successfully used for refocusing light field images selected from open-source datasets as well as captured using commercial light field cameras faster than other released light field refocusing programs. Considering its fast processing speed and user-friendly operations, OpenRefocus can be a useful tool for light field refocusing in many light field imaging applications. [ABSTRACT FROM AUTHOR]

The spatial positions of multiple fragments formed by projectile explosion have random uncertainty dispersion state, so it is difficult to recognize each fragment and measure its spatial position when multiple fragments reach a certain area at the same time. In order to solve the problem, we propose a test method for measuring the spatial positions of multiple fragments based on dual light field intersection imaging mechanism, study refocusing principle of micro-lens array and four-dimensional light field imaging information, and establish the analytical model of spatial positions of multiple fragments. We set up the uncertainty error evaluation function of two light field cameras intersection test method from the perspective of engineering design, and give the uncertainty analysis method under engineering model. The influence of various factors on the uncertainty of test results under the engineering model is analyzed by calculation. Combined with the actual test data of the wooden target, the results show that the fragment spatial position test method based on dual light field intersection imaging mechanism is feasible and scientific. [ABSTRACT FROM AUTHOR]

Light Field (LF) imaging is a plenoptic data collection method enabling a wide variety of image post-processing such as 3D extraction, viewpoint change and digital refocusing. Moreover, LF provides the capability to capture rich information about a scene, e.g., texture, geometric information, etc. Therefore, a quality assessment model for LF images is needed and poses significant challenges. Many LF Image Quality Assessment (LF-IQA) metrics have been recently presented based on the unique characteristics of LF images. The state-of-the-art objective assessment metrics have taken into account the image content and human visual system such as SSIM and IW-SSIM. However, most of these metrics are designed for images and video with natural content. Additionally, other models based on the LF characteristics (e.g., depth information, angle information) trade high performance for high computational complexity, along with them possessing difficulties of implementation for LF applications due to the immense data requirements of LF images. Hence, this paper presents a novel content-adaptive LF-IQA metric to improve the conventional LF-IQA performance that is also low in computational complexity. The experimental results clearly show improved performance compared to conventional objective IQA metrics, and we also identify metrics that are well-suited for LF image assessment. In addition, we present a comprehensive content-based feature analysis to determine the most appropriate feature that influences human visual perception among the widely used conventional objective IQA metrics. Finally, a rich LF dataset is selected from the EPFL dataset, allowing for the study of light field quality by qualitative factors such as depth (wide and narrow), focus (background or foreground) and complexity (simple and complex). [ABSTRACT FROM AUTHOR]