Search

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography’s limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

The handheld, do-it-yourself ptychographic whole slide scanner for high-throughput digital pathology applications.

Ptychography is an enabling coherent diffraction imaging technique for both fundamental and applied sciences. Its applications in optical microscopy, however, fall short for its low imaging throughput and limited resolution. Here, we report a resolution-enhanced parallel coded ptychography technique achieving the highest numerical aperture and an imaging throughput orders of magnitude greater than previous demonstrations. In this platform, we translate the samples across the disorder-engineered surfaces for lensless diffraction data acquisition. The engineered surface consists of chemically etched micron-level phase scatters and printed sub-wavelength intensity absorbers. It is designed to unlock an optical space with spatial extent (x, y) and frequency content (kx, ky) that is inaccessible using conventional lens-based optics. To achieve the best resolution performance, we also report a new coherent diffraction imaging model by considering both the spatial and angular responses of the pixel readouts. Our low-cost prototype can directly resolve 308-nm linewidth on the resolution target without aperture synthesizing. Gigapixel high-resolution microscopic images with a 240-mm^2 effective field of view can be acquired in 15 seconds. For demonstrations, we recover slow-varying 3D phase objects with many 2{\pi} wraps, including optical prism and convex lens. The low-frequency phase contents of these objects are challenging to obtain using other existing lensless techniques. For digital pathology applications, we perform accurate virtual staining by using the recovered phase as attention guidance in a deep neural network. Parallel optical processing using the reported technique enables novel optical instruments with inherent quantitative nature and metrological versatility., Comment: 17 pages; 7 figures

In this study, 30 subjects were exposed to different combinations of air temperature (Ta : 24, 27, and 30°C) and CO2 level (8000, 10 000, and 12 000 ppm) in a high-humidity (RH: 85%) underground climate chamber. Subjective assessments, physiological responses, and cognitive performance were investigated. The results showed that as compared with exposure to Ta = 24°C, exposure to 30°C at all CO2 levels caused subjects to feel uncomfortably warm and experience stronger odor intensity, while increased mental effort and greater intensity of acute health symptoms were reported. However, no significant effects of Ta on task performance or physiological responses were found. This indicated that subjects had to exert more effort to maintain their performance in an uncomfortably warm environment. Increasing CO2 from 8000 to 12 000 ppm at all Ta caused subjects to report higher rates of headache, fatigue, agitation, and feeling depressed, although the results were statistically significant only at 24 and 27°C. The text typing performance and systolic blood pressure (SBP) decreased significantly at this exposure, whereas diastolic blood pressure (DBP) and thermal discomfort increased significantly. These effects suggest higher arousal/stress. No significant interaction effect of Ta and CO2 concentration on human responses was identified.

The applications of conventional ptychography are limited by its relatively low resolution and throughput in the visible light regime. The new development of coded ptychography (CP) has addressed these issues and achieved the highest numerical aperture for large-area optical imaging in a lensless configuration. A high-quality reconstruction of CP relies on precise tracking of the coded sensor’s positional shifts. The coded layer on the sensor, however, prevents the use of cross correlation analysis for motion tracking. Here we derive and analyze the motion tracking model of CP. A novel, to the best of our knowledge, remote referencing scheme and its subsequent refinement pipeline are developed for blind image acquisition. By using this approach, we can suppress the correlation peak caused by the coded surface and recover the positional shifts with deep sub-pixel accuracy. In contrast with common positional refinement methods, the reported approach can be disentangled from the iterative phase retrieval process and is computationally efficient. It allows blind image acquisition without motion feedback from the scanning process. It also provides a robust and reliable solution for implementing ptychography with high imaging throughput. We validate this approach by performing high-resolution whole slide imaging of bio-specimens.

Blu-ray drive is an engineering masterpiece that integrates disc rotation, pickup head translation, and three lasers in a compact and portable format. Here we integrate a blood-coated image sensor with a modified Blu-ray drive for high-throughput cytometric analysis of various bio-specimens. In this device, samples are mounted on the rotating Blu-ray disc and illuminated by the built-in lasers from the pickup head. The resulting coherent diffraction patterns are then recorded by the blood-coated image sensor. The rich spatial features of the blood-cell monolayer help down-modulate the object information for sensor detection, thus forming a high-resolution computational bio-lens with a theoretically unlimited field of view. With the acquired data, we develop a lensless coherent diffraction imaging modality termed rotational ptychography for image reconstruction. We show that our device can resolve the 435 nm linewidth on the resolution target and has a field of view only limited by the size of the Blu-ray disc. To demonstrate its applications, we perform high-throughput urinalysis by locating disease-related calcium oxalate crystals over the entire microscope slide. We also quantify different types of cells on a blood smear with an acquisition speed of ~10,000 cells per second. For in vitro experiment, we monitor live bacterial cultures over the entire Petri dish with single-cell resolution. Using biological cells as a computational lens could enable new intriguing imaging devices for point-of-care diagnostics. Modifying a Blu-ray drive with the blood-coated sensor further allows the spread of high-throughput optical microscopy from well-equipped laboratories to citizen scientists worldwide.

Traditional microbial detection methods often rely on the overall property of microbial cultures and cannot resolve individual growth event at high spatiotemporal resolution. As a result, they require bacteria to grow to confluence and then interpret the results. Here, we demonstrate the application of an integrated ptychographic sensor for lensless cytometric analysis of microbial cultures over a large scale and with high spatiotemporal resolution. The reported device can be placed within a regular incubator or used as a standalone incubating unit for long-term microbial monitoring. For longitudinal study where massive data are acquired at sequential time points, we report a new temporal-similarity constraint to increase the temporal resolution of ptychographic reconstruction by 7-fold. With this strategy, the reported device achieves a centimeter-scale field of view, a half-pitch spatial resolution of 488 nm, and a temporal resolution of 15-second intervals. For the first time, we report the direct observation of bacterial growth in a 15-second interval by tracking the phase wraps of the recovered images, with high phase sensitivity like that in interferometric measurements. We also characterize cell growth via longitudinal dry mass measurement and perform rapid bacterial detection at low concentrations. For drug-screening application, we demonstrate proof-of-concept antibiotic susceptibility testing and perform single-cell analysis of antibiotic-induced filamentation. The combination of high phase sensitivity, high spatiotemporal resolution, and large field of view is unique among existing microscopy techniques. As a quantitative and miniaturized platform, it can improve studies with microorganisms and other biospecimens at resource-limited settings., 18 pages, 6 figures

Ptychography-based lensless on-chip microscopy enables high-throughput imaging by retrieving the missing phase information from intensity measurements. Numerous reconstruction algorithms for ptychography have been proposed, yet only a few incremental algorithms can be extended to lensless on-chip microscopy because of large-scale datasets but limited computational efficiency. In this paper, we propose the use of accelerated proximal gradient methods for blind ptychographic phase retrieval in lensless on-chip microscopy. Incremental gradient approaches are adopted in the reconstruction routine. Our algorithms divide the phase retrieval problem into sub-problems involving the evaluation of proximal operator, stochastic gradient descent, and Wirtinger derivatives. We benchmark the performances of accelerated proximal gradient, extended ptychographic iterative engine, and alternating direction method of multipliers, and discuss their convergence and accuracy in both noisy and noiseless cases. We also validate our algorithms using experimental datasets, where full field of view measurements are captured to recover the high-resolution complex samples. Among these algorithms, accelerated proximal gradient presents the overall best performance regarding accuracy and convergence rate. The proposed methods may find applications in ptychographic reconstruction, especially for cases where a wide field of view and high resolution are desired at the same time.

Conventional ptychography translates an object through a localized probe beam to widen the field of view in real space. Fourier ptychography translates the object spectrum through a pupil aperture to expand the Fourier bandwidth in reciprocal space. Here we report an imaging modality, termed synthetic aperture ptychography (SAP), to get the best of both techniques. In SAP, we illuminate a stationary object using an extended plane wave and translate a coded image sensor at the far field for data acquisition. The coded layer attached on the sensor modulates the object exit waves and serves as an effective ptychographic probe for phase retrieval. The sensor translation process in SAP synthesizes a large complex-valued wavefront at the intermediate aperture plane. By propagating this wavefront back to the object plane, we can widen the field of view in real space and expand the Fourier bandwidth in reciprocal space simultaneously. We validate the SAP approach with transmission targets and reflection silicon microchips. A 20-mm aperture was synthesized using a 5-mm sensor, achieving a fourfold gain in resolution and 16-fold gain in field of view for object recovery. In addition, the thin sample requirement in ptychography is no longer required in SAP. One can digitally propagate the recovered exit wave to any axial position for post-acquisition refocusing. The SAP scheme offers a solution for far-field sub-diffraction imaging without using lenses. It can be adopted in coherent diffraction imaging setups with radiation sources from visible light, extreme ultraviolet, and X-ray, to electron.

Whole slide imaging (WSI) has moved digital pathology closer to diagnostic practice in recent years. Due to the inherent tissue topography variability, accurate autofocusing remains a critical challenge for WSI and automated microscopy systems. The traditional focus map surveying method is limited in its ability to acquire a high degree of focus points while still maintaining high throughput. Real-time approaches decouple image acquisition from focusing, thus allowing for rapid scanning while maintaining continuous accurate focus. This work reviews the traditional focus map approach and discusses the choice of focus measure for focal plane determination. It also discusses various real-time autofocusing approaches including reflective-based triangulation, confocal pinhole detection, low-coherence interferometry, tilted sensor approach, independent dual sensor scanning, beam splitter array, phase detection, dual-LED illumination, and deep-learning approaches. The technical concepts, merits, and limitations of these methods are explained and compared to those of a traditional WSI system. This review may provide new insights for the development of high-throughput automated microscopy imaging systems that can be made broadly available and utilizable without loss of capacity.

Fourier ptychographic microscopy (FPM) is a computational approach geared towards creating high-resolution and large field-of-view images without mechanical scanning. Acquiring color images of histology slides often requires sequential acquisitions with red, green, and blue illuminations. The color reconstructions often suffer from coherent artifacts that are not presented in regular incoherent microscopy images. As a result, it remains a challenge to employ FPM for digital pathology applications, where resolution and color accuracy are of critical importance. Here we report a deep learning approach for performing unsupervised image-to-image translation of FPM reconstructions. A cycle-consistent adversarial network with multiscale structure similarity loss is trained to perform virtual brightfield and fluorescence staining of the recovered FPM images. In the training stage, we feed the network with two sets of unpaired images: (1) monochromatic FPM recovery and (2) color or fluorescence images captured using a regular microscope. In the inference stage, the network takes the FPM input and outputs a virtually stained image with reduced coherent artifacts and improved image quality. We test the approach on various samples with different staining protocols. High-quality color and fluorescence reconstructions validate its effectiveness.

We discuss two compact, cost-effective, and field-portable ptychographic lensless imaging platforms for quantitative microscopy. In the first implementation, we use a low-cost galvo scanner to rapidly scan an unknown laser speckle pattern on the object. To address the positioning repeatability and accuracy issues, we directly recover the positional shifts of the speckle pattern based on the phase correlation of the captured images. To bypass the resolution limit set by the imager pixel size, we employ a sub-sampled ptychographic phase retrieval process to recover the complex object. In the second implementation, we place a thin diffuser in between the object and the image sensor for light wave modulation. By blindly scanning the unknown diffuser to different x-y positions, we acquire a sequence of modulated intensity images for quantitative object recovery. Different from previous ptychographic implementations, we employ a unit magnification configuration with a Fresnel number of ~50,000, which is orders of magnitude higher than previous ptychographic setups. The unit magnification configuration allows us to have the entire sensor area, 6.4 mm by 4.6 mm, as the imaging field of view. The ultra-high Fresnel number enables us to directly recover the positional shift of the diffuser in the phase retrieval process. In this second implementation, we use a low-cost, DIY scanning stage to perform blind diffuser modulation. We further employ an up-sampling phase retrieval scheme to bypass the resolution limit set by the imager pixel size and demonstrate a half-pitch resolution of 0.78 µm. For both implementations, we validate the imaging performance via various biological samples. The reported platforms provide cost-effective and turnkey solutions for large field-of-view, high-resolution, and quantitative on-chip microscopy. They are adaptable for a wide range of point-of-care-, global-health-, and telemedicine-related applications.

Recent advancements in whole slide imaging (WSI) have moved pathology closer to digital practice. Existing systems require precise mechanical control, and the cost is prohibitive for most individual pathologists. Here we report a low-cost and high-throughput WSI system termed OpenWSI. The reported system is built using off-the-shelf components, including a programmable LED array, a photographic lens, and a low-cost computer numerical control (CNC) router. Different from conventional WSI platforms, our system performs real-time single-frame autofocusing using color-multiplexed illumination. For axial positioning control, we perform coarse adjustment using the CNC router and precise adjustment using the ultrasonic motor ring in the photographic lens. By using a 20X objective lens, we show that the OpenWSI system has a resolution of ∼ 0.7 µ m . It can acquire whole slide images of a 225 m m 2 region in ∼ 2 m i n s with throughput comparable to existing high-end platforms. The reported system offers a turnkey solution to transform the high-end WSI platforms into one that can be made broadly available and utilizable without loss of capacity.

We report a novel lensless on-chip microscopy platform based on near-field blind ptychographic modulation. In this platform, we place a thin diffuser in between the object and the image sensor for light wave modulation. By blindly scanning the unknown diffuser to different x-y positions, we acquire a sequence of modulated intensity images for quantitative object recovery. Different from previous ptychographic implementations, we employ a unit magnification configuration with a Fresnel number of ∼50 000, which is orders of magnitude higher than those of previous ptychographic setups. The unit magnification configuration allows us to have the entire sensor area, 6.4 mm by 4.6 mm, as the imaging field of view. The ultra-high Fresnel number enables us to directly recover the positional shift of the diffuser in the phase retrieval process, addressing the positioning accuracy issue plaguing regular ptychographic experiments. In our implementation, we use a low-cost, DIY scanning stage to perform blind diffuser modulation. Precise mechanical scanning that is critical in conventional ptychography experiments is no longer needed in our setup. We further employ an up-sampling phase retrieval scheme to bypass the resolution limit set by the imager pixel size and demonstrate a half-pitch resolution of 0.78 μm. We validate the imaging performance via in vitro cell cultures, transparent and stained tissue sections, and a thick biological sample. We show that the recovered quantitative phase map can be used to perform effective cell segmentation of a dense yeast culture. We also demonstrate 3D digital refocusing of the thick biological sample based on the recovered wavefront. The reported platform provides a cost-effective and turnkey solution for large field-of-view, high-resolution, and quantitative on-chip microscopy. It is adaptable for a wide range of point-of-care-, global-health-, and telemedicine-related applications.

We report an angle-tilted, wavelength-multiplexed ptychographic modulation approach for multispectral lensless on-chip microscopy. In this approach, we illuminate the specimen with lights at five wavelengths simultaneously. A prism is added at the illumination path for spectral dispersion. Thus, lightwaves at different wavelengths hit the specimen at slightly different incident angles, breaking the ambiguities in mixed-state ptychographic reconstruction. At the detection path, we place a thin diffuser between the specimen and the monochromatic image sensor for encoding the spectral information into 2D intensity measurements. By scanning the sample to different x − y positions, we acquire a sequence of monochromatic images for reconstructing the five complex object profiles at the five wavelengths. An up-sampling procedure is integrated into the recovery process to bypass the resolution limit imposed by the imager pixel size. We demonstrate a half-pitch resolution of 0.55 µm using an image sensor with 1.85 µm pixel size. We also demonstrate quantitative and high-quality multispectral reconstructions of stained tissue sections for digital pathology applications.