Search

Alzheimer’s disease (AD) is a major risk for the aging population. The pathological hallmarks of AD—an abnormal deposition of amyloid β-protein (Aβ) and phosphorylated tau (pTau)—have been demonstrated in the retinas of AD patients, including in prodromal patients with mild cognitive impairment (MCI). Aβ pathology, especially the accumulation of the amyloidogenic 42-residue long alloform (Aβ42), is considered an early and specific sign of AD, and together with tauopathy, confirms AD diagnosis. To visualize retinal Aβ and pTau, state-of-the-art methods use fluorescence. However, administering contrast agents complicates the imaging procedure. To address this problem from fundamentals, this dissertation performed ex vivo studies to develop a label-free hyperspectral imaging method to detect the spectral signatures of Aβ42 and pS396-Tau. A deep learning framework was developed to predict their abundance in retinal cross-sections and transform a label-free HSI image to either a DAB or an immunofluorescent stained image in high accuracy. For the first time, we reported the spectral signature of pTau and provided a direct validation through immunostaining. For small incision in vivo imaging, optical endoscopes are mostly limited by two-dimensional views or very small number of three-dimensional (3D) views of pathological sites, and are intrinsically low in resolution caused by the limited fiber cores. The dissertation demonstrated a flexible light field endoscopy (Flex- LFE) imaging system capable of capturing depth information in a single shot. To address the resolution challenges inherent in endoscopic imaging, an AI-powered super-resolution pipeline is developed to enhance the quality of reconstructed images.Additionally, this work explored the utilization of tunable image-mapping optical coherence tomography (TIM-OCT) to further advance in imaging of the retina. While most current OCT devices require extensive scanning, by combining phase-only spatial light modulators with spectral domain OCT, TIM-OCT achieves tailored imaging performance and enables “eye motion freeze” snapshot imaging. Computational spectroscopic analysis is discussed to extract spectral signatures.This dissertation demonstrates the potential of AI-driven optical imaging systems in addressing critical challenges in biomedical imaging, with a particular focus on the early detection of AD. The findings are expected to lay the groundwork for label-free detection of AD.

Imaging systems capture light rays contain rich information, which can be described by the plenoptic function, ?(?, ?, ?, ?, ?, ?, ?)—where ?, ?, ? represent spatial coordinates; ?, ? denote emittance angles; ? signifies wavelength; and ? denotes time. Given a finite photon budget, it is crucial for an imaging system to maximize the information yield from each captured image. Yet, traditional cameras only capture two-dimensional spatial data (?, ?), neglecting the wealth of information.Capturing multi-dimensional information presents significant challenges, primarily due to the complexity of mapping high-dimensional datacubes onto a two-dimensional detector array. This mapping process introduces a fundamental trade-off among various axes of information, such as spatial, angular, and spectral dimensions, which can adversely affect imaging speed, resolution, and other critical parameters. Balancing of these factors often leads to compromises in one aspect to enhance another, pose a significant challenge in designing and implementing a multi-dimensional imaging system.In response to these challenges, this dissertation presents three innovative multi-dimensional optical imaging systems. The first system, snapshot hyperspectral light field imaging utilizing image mapping spectrometer (LF-IMS), represents a five-dimensional (?, ?, ?, ?, ?) imaging system. It is uniquely designed to maintain full light throughput, enabling the capture of detailed three-dimensional spatial and spectral information without sacrificing efficiency. The second system, snapshot hyperspectral light field tomography (Hyper-LIFT), leverages compressed sensing to facilitate five-dimensional (?, ?, ?, ?, ?) imaging. This approach significantly alleviates the tradeoff between different dimensions of information, allowing for capturing an input scene with a more compact sensor, thereby greatly reducing the volume of data generated during image acquisition. The third system, the tunable image projection spectrometer (TIPS), is a Fourier-domain line-scan hyperspectral imager with a tunable compression ratio. Compared to state-of-the-art spatial-domain pushbroom hyperspectral cameras, TIPS requires much fewer measurements and provides a higher light throughput.Furthermore, this dissertation will explore the impact of optical aberrations on the image quality in light field imaging, providing insights into how these imperfections influence the overall imaging performance. A lens design pipeline is proposed to mitigate key aberrations, and its effectiveness is demonstrated through the design of a light field endoscope. Additionally, a rapid calibration method has been proposed for a compact hyperspectral camera, termed the image mapping spectrometer (IMS), reducing the calibration time from weeks to hours.

Meta-lens has successfully been developed for a variety of optical functions. We demonstrate a light-field edge detection imaging system with a gallium nitride achromatic meta-lens array. It enables edge detection from one dimension to three dimensions. The designed meta-lens array consists of 60 by 60 achromatic meta-lenses, which operate in the visible range from 400 to 660 nm. All of the light field information of objects in the scene can be captured and computed. The focused edge images from one dimension to three dimensions are extracted with depth estimation by image rendering. Three dimensions edge detection is two dimensions edge imaging with depth information. The focused edge images can be obtained by the sub-image reconstruction of the light field image. Our multidimensional edge detection system by achromatic meta-lens array brings novel advantages, such as broadband detection, data volume reduction, and device miniaturization capacity. Results of our experiments show new insight into applications of biological diagnose and robotic vision.

We present a new category of computational ultrafast imaging technique, light field tomography (LIFT), which can perform 3D snapshot transient (time-resolved) imaging at an unprecedented frame rate with full-fledged light field imaging capabilities including depth retrieval, post-capture refocusing, and extended depth of field. As a niche application, we demonstrated real-time non-line-of-sight imaging of fast-moving hidden objects, which is previously impossible without the presented technique. Moreover, we showcased 3D imaging of fiber-guided light propagation along a twisted path and the capability of resolving extended 3D objects.

Terahertz focal plane array imaging methods, direct camera imaging and conventional light field imaging methods are incapable of resolving and separating layers of multilayer objects. In this paper, for the purpose of fast, high-resolution and layer-resolving imaging of multilayer structures with different reflection characteristics, a novel angular intensity filtering (AIF) method based on terahertz light-field imaging is purposed. The method utilizes the extra dimensional information from the 4D light field and the reflection characteristics of the imaging object, and the method is capable to resolve and reconstruct layers individually. The feasibility of the method is validated by experiment on both “idealized” and “practical” multilayer samples, and the advantages in performance of the method are proven by quantitative comparison with conventional methods.

Microlenses can be widely used in integrated micro-optical systems. However, in some special applications, such as light field imaging systems, multifocal microlens arrays (MLA) are expected to improve imaging resolution. For the fabrication of multifocal MLA, the traditional fabrication method is no longer applicable. To solve this problem, a fabrication method of multifocal MLA by a one step exposure process is proposed. Through the analyses and research of photoresist AZ9260, the nonlinear relationship between exposure dose and exposure depth is established. In the design of the mask, the mask pattern is corrected according to the nonlinear relationship to obtain the final mask. The continuous surface of the multifocal MLA is fabricated by the mask moving exposure. The experimental results show that the prepared multifocal MLA has high filling factor and surface fidelity. What is more, this method is simple and efficient to use in practical applications.

A silica-based embedded aluminum grating polarizer suitable for a near- and mid-infrared light field camera is investigated to fully utilize the direction information recorded by the detector in the light field camera. A grating with good polarizer performance was obtained by using the finite-difference time-domain (FDTD) method to simulate and optimize the structural parameters affecting the performance. The optimized structure reaches TM transmission efficiency exceeding 84% and extinction ratios greater than 27.9 dB in the 1.5–5.5 μm band. The optimized grating is suitable for a wide incident angle range of 0°−70° and for infrared polarization light field imaging. Keywords: The embedded grating, Light field imaging, Finite-difference time-domain method, Extinction ratios, TM wave transmission efficiency

In this paper, we report a dispersion-eliminated coded-aperture light field (DECALF) imaging system based on digital micromirror devices (DMDs). Using a dual-DMD design to compensate for dispersion in the entire visible spectrum, the DECALF imaging system captures 1280x1024x5x5 color light field images at 20 Hz. we apply it to tracking movements of normal and disease-model zebrafish.

We built a fiber-bundle light-field imaging system for endoscopic depth imaging. The intrinsically low resolution of the fiber-based light field imaging was greatly improved by training a deep neural network using high-resolution light field images.

In this Letter, we investigate a snapshot spectral-polarimetric-volumetric imaging (SSPVI) system using a single detector. Through compressed acquisition and reconstruction, SSPVI can achieve spectral imaging ( x , y , λ ) , polarization imaging ( x , y , ψ , χ ) , and light field imaging ( x , y , θ , φ ) simultaneously. The newly discovered performance is showcased by attaining the spectral-polarimetric-volumetric video and different laboratory accuracy experiments. These never-seen-before capacities of the camera open new prospects for many applications, such as biological analysis, object recognition, and remote sensing.

3D localization of point source is widely used in many fields, such as bioimaging and autonomous driving fields. However, the localization is hard to perform under scattering conditions because of the diffuse effect of the scattering. We propose a novel method for 3D localization of point source under scattering conditions based on light field imaging and deep learning by only one shot. First, we introduce the description of the point source in a light field wise and how to localize a point source by its light field. On the basis, we elaborate on the effect of scattering on a light field and how to retrieve the location of a point source from a light field with scattering. Then, the effect of aberration on a light field will be introduced. We also build an artificial scene and a deep learning framework to perform a 3D localization practically, and the feasibility and accuracy of our method have been evaluated.

The flame temperature measurement is significant to study flame structure. However, during the detection process using light field imaging technology, the influence of radiative properties on flame distribution at different positions is hard to evaluate. Therefore, this study simulates the light field imaging of section flames to analyze the influence of radiative properties on flame distribution. A tomographic image is reconstructed from three-dimensional light field imaging, which is evaluated as similar to the section flame images. Then the temperature reconstructed from the tomographic image is compared to the known temperature, with a small relative error, under 10%. In addition, the influence of radiative properties on the accuracy of reconstructed temperature is also analyzed.

A microlens array (MLA) is a key optical element for four-dimensional light-field analysis in a light-field camera. MLA manufacturing errors affect light-field imaging, but their effects, especially on the physical imaging process and image-degradation mechanism, have not entirely been studied. In this paper, we develop a manufacturing-error model for MLA and quantitatively analyze raw images and refocused images using image quality evaluation indexes. The results indicate that manufacturing errors cause changes in image features including brightness, resolution, and spot position. The image-features undergo changes and the degree of degradation caused by different error types showed great differences.

We present an improvement of depth-of-field in light field imaging system using multi-focal micro-lens array. By adjusting etch rate at same wafer, we implement a multi-focus micro-lens array and demonstrate light field imaging system.

The paper discusses the light efficiency and SNR of light field imaging systems in comparison to classical 2D imaging. In order to achieve this goal, I define the "equivalent f-number" as a concept of capturing the light gathering ability of the light field system. Since the f-number, in classical imaging systems, is conceptually also linked with the depth-of-field, I discuss an appropriate depth-of-field interpretation for light field systems.

By combining a high-resolution image from a standard camera with a low-resolution light-field image from a lenslet array, we numerically reconstruct a high-resolution light-field image. We experimentally demonstrate the method by creating a high-definition 3D image of a human cheek cell with a commercially available microscope.

Coded-aperture light field (CALF) imaging can record four-dimensional information of incident light rays with a high angular resolution while retaining a camera’s full pixel count. However, existing systems are limited by either imaging speeds and image contrasts of liquid-crystal spatial light modulators or by severe dispersion to broadband light of digital micromirror devices (DMDs), both of which hinder CALF’s widespread applications. Here, we overcome these limitations by developing dispersion-eliminated (DE) CALF imaging. Using a dual-DMD design to compensate for dispersion in the entire visible spectrum, the DECALF imaging system captures 1280 × 1024 × 5 × 5 ( x , y , θ , φ ) color light field images at 20 Hz. Using static and dynamic three-dimensional (3D) color scenes, we experimentally demonstrate multi-perspective viewing, digital refocusing, and 3D tracking of the DECALF imaging system. We also apply it to the imaging and analyses of escape behaviors of freely moving normal and disease-model zebrafish.

A relatively simple method to improve the image resolution of light field based on a liquid crystal (LC) microlens doped with multi-walled carbon nanotubes (MWCNTs) was developed and evaluated. As the nanoparticles were doped in LC, its electro-optical features could enhance, leading to a short response time compared to the pure LC microlens. With the maximum use of the proposed LC microlens, a method combining aperiodicity extraction and weighted average algorithm was adopted to realize the high-resolution light field imaging. The aperiodicity extraction method was proposed, which could effectively improve resolution of view angle image. For synthesizing the full resolution image at 0 Vrms and the extracted view angle image of light field imaging at 2.0 Vrms, the final high-resolution light field imaging could be obtained in a short time by weighted average algorithm. In this way, the common problem of low resolution in light field imaging could be solved. This proposed method was in good agreement with our experimental results. And it was also in line with the development of the trend of the smart imaging sensor combining algorithm with hardware.

In terahertz imaging systems based on Gaussian beam active illumination and focal plane array detectors, severe image distortion has been observed, which significantly reduces the resolving power of the imaging system. To solve this problem, a novel computational method, Light Field Imaging (LFI), has been introduced for terahertz imaging. A conventional transmission-type terahertz imaging system based on a gas-pumped terahertz source and terahertz Focal Plane Array Detectors (FPA) arrays is established to analyze the problem of image distortion. An experimental virtual camera array terahertz LFI system is also established. With the acquisition and reconstruction of synthetic aperture terahertz light fields, the improvement on resolving power and SNR performance have been validated.

This work proposes a novel compressive spectral light field imaging architecture along with a tensor-based reconstruction algorithm to estimate the four-dimensional image source from a set of compressed measurements.

Light field microscopy through bare optical fiber bundles paves the way for depth-resolved fluorescence microendoscopy., Optical fiber bundle microendoscopes are widely used for visualizing hard-to-reach areas of the human body. These ultrathin devices often forgo tunable focusing optics because of size constraints and are therefore limited to two-dimensional (2D) imaging modalities. Ideally, microendoscopes would record 3D information for accurate clinical and biological interpretation, without bulky optomechanical parts. Here, we demonstrate that the optical fiber bundles commonly used in microendoscopy are inherently sensitive to depth information. We use the mode structure within fiber bundle cores to extract the spatio-angular description of captured light rays—the light field—enabling digital refocusing, stereo visualization, and surface and depth mapping of microscopic scenes at the distal fiber tip. Our work opens a route for minimally invasive clinical microendoscopy using standard bare fiber bundle probes. Unlike coherent 3D multimode fiber imaging techniques, our incoherent approach is single shot and resilient to fiber bending, making it attractive for clinical adoption.

Volume hologram, which has unique characteristics such as wavelength and angular selectivity, is a powerful tool for enabling computational imaging. For example, light-field three-dimensional imaging can be realized with a thin, flat, and transparent material by utilizing the volume hologram. This report mainly introduces the light-field imaging system utilizing volume hologram with describing a calibration method and an experimental result. The talk corresponding to this report also addresses other imaging applications of the volume hologram.

Combined with the light-field imaging technique, the Landweber method is applied to the reconstruction of three-dimensional (3D) temperature distribution in absorbing media theoretically and experimentally. In the theoretical research, simulated exit radiation intensities on the boundary of absorbing media according to the computing model of light field are employed as inputs for inverse analysis. Compared with the commonly used iterative methods, i.e., the least-square QR decomposition method and algebraic reconstruction technique (ART), the Landweber method can produce reconstruction results with better quality and less computational time. Based on the numerical study, an experimental investigation is conducted to validate the suitability of the proposed method. The temperature distribution of the ethylene diffusion flame is reconstructed by using the Landweber method from the flame image captured by a light-field camera. Good agreement was found between the reconstructed temperature distribution and the measured temperature data obtained by a thermocouple. All the experimental results demonstrate that the temperature distribution of ethylene flame can be reconstructed reasonably by using the Landweber method combined with the light-field imaging technique, which is proven to have potential for the use in noncontract measurement of temperature distribution in practical engineering applications.

Light field imaging spectrometer is a snapshot spectrometer which can capture spatial and spectral information of objects in one snapshot. Chromatic aberration of the fore-optics will cause deviation of the imaging position of the filter-array. The effect of the chromatic aberration of the fore-optics of a light field imaging spectrometer is analyzed and a correction method is presented.

In this paper, a method for measuring the non-uniform refractive index field based on the light field imaging technique is proposed. First, the light field camera is used to collect the four-dimensional light field data, and then the light field data is decoded according to the light field imaging principle to obtain image sequences with different acquisition angles of the refractive index field. Subsequently PIV (Particle Image Velocimetry) technique is used to extract ray offset of each image. Finally, the distribution of non-uniform refractive index field can be calculated by inversing the deflection of light rays. Compared with traditional optical methods which require multiple optical detectors from multiple angles to synchronously collect data, the method proposed in this paper only needs a light field camera and shoot once. The effectiveness of the method has been verified by the experiment which quantitatively measures the distribution of the refractive index field above the flame of the alcohol lamp.

An overlapped micro-lens array containing 222,762 lenslets is proposed and designed to increase the utilization of image sensors in light field imaging systems. The array is made by micro-injection moulding, and the mould is processed by ultra-precision milling to achieve high precision and a low manufacturing cost. To obtain a compact configuration of each image under a micro-lens, a customized square aperture is introduced into the light path. The experiment shows that compared with the conventional design, the overlapped micro-lens array has improved the fill factor from π/4 to 1, which largely increases the utilization of the image sensors.

In the recent computational imaging and display literature, light field cameras and displays are being treated distinctly, without an obvious connection. Despite significant differences in the signal processing tools employed in compressive light field imaging systems, we argue that there is a natural duality between these systems. Following Gabriel Lippmann's vision of integrated light field cameras and displays, we develop an intuitive interpretation of the optical systems and compressive optimization schemes employed in modern compressive light field imaging systems.

We present a fabrication method of microlens array with large-area/low-cost. We also produce a hand-crafted light field camera using the customized microlens array and demonstrate a light field imaging feature.

Light field imaging involves capturing both angular and spatial distribution of light; it enables new capabilities, such as post-capture digital refocusing, camera aperture adjustment, perspective shift, and depth estimation. Micro-lens array (MLA) based light field cameras provide a cost-effective approach to light field imaging. There are two main limitations of MLA-based light field cameras: low spatial resolution and narrow baseline. While low spatial resolution limits the general purpose use and applicability of light field cameras, narrow baseline limits the depth estimation range and accuracy. In this paper, we present a hybrid stereo imaging system that includes a light field camera and a regular camera. The hybrid system addresses both spatial resolution and narrow baseline issues of the MLA-based light field cameras while preserving light field imaging capabilities., Comment: Machine Vision and Applications

Plenoptic cameras are used for capturing flames in studies of high-temperature phenomena. However, simulations of plenoptic camera models can be used prior to the experiment improve experimental efficiency and reduce cost. In this work, microlens arrays, which are based on the established light field camera model, are optimized into a hexagonal structure with three types of microlenses. With this improved plenoptic camera model, light field imaging of static objects and flame are simulated using the calibrated parameters of the Raytrix camera (R29). The optimized models improve the image resolution, imaging screen utilization, and shooting range of depth of field.

Based on the principium of light field imaging, there designed a objective lens and a microlens array for gathering the light field feature, the homologous ZEMAX models was also be built. Then all the parameters were optimized using ZEMAX and the simulation image was given out. It pointed out that the position relationship between the objective lens and the microlens array had a great affect on imaging, which was the guidance when developing a prototype.

Recently, intensive research on microlens array (MLA) was undertaken, which involved intertwining with the light-field imaging in order to obtain four-dimensional information. Although several fabrication approaches for MLA attempted to achieve high density and precision geometry, further progress is necessary. In this study, we present a cost-effective fabrication strategy for geometrically tunable polymer MLA with extremely low sag height (~3 μm) through improved MEMS wet etching process. Additionally, we assemble a hand-crafted light-field camera by integrating the elaborately customized MLA with a commercial digital camera. Finally, we demonstrate representative light-field imaging features including refocusing and all-in focusing image from a single exposure.