Search

OptoMechanical Modulation Tomography (OMMT) exploits compressed sensing to reconstruct high resolution microscopy volumes from fewer measurement images compared to exhaustive section sampling in conventional light sheet microscopy. Nevertheless, the volumetric reconstruction process is computationally expensive, making it impractically slow to use on large-size images, and prone to generating visual artefacts. Here, we propose a reconstruction approach that uses a 1+2D Total Variation regularization that does not generate such artefacts and is amenable to efficient implementation using parallel computing. We evaluate our method for accuracy and scaleability on simulated and experimental data. Using a high quality, but computationally expensive, Plug-and-Play (PnP) method that uses the BM4D denoiser as a benchmark, we observe that our approach offers an advantageous trade-off between speed and accuracy.

Many applications rely on thermal imagers to complement or replace visible light sensors in difficult imaging conditions. Recent advances in machine learning have opened the possibility of analyzing or enhancing images, yet these methods require large annotated databases. Training approaches that leverage data augmentation via simulated and synthetically-generated images could offer promising prospects. Here, we report on a method that uses generative adversarial nets (GANs) to synthesize images of a complementary contrast. Starting from a dual-modality dataset of co-registered visible and thermal images, we trained a GAN to generate synthetic thermal images from visible images and vice versa. Our results show that the procedure yields sharp synthesized images that might be used to augment dual-modality datasets or assist in visual interpretation, yet are also subject to the limitations imposed by contrast independence between thermal and visible images.

Fluorescence laser-scanning microscopy is a well-established imaging technique in biology, available in many imaging facilities to investigate structures within live animal embryos such as zebrafish. Laser scanning microscopes (LSM) are limited when used to study dynamic heart morphology or function. Despite their ability to resolve static cardiac structures, the fast motion of the beating heart introduces severe artifacts in the scanned images and gating the acquisitions to the heartbeat is difficult to implement on traditional microscopes. Furthermore, although alternative high-speed imaging instruments exist, they are not widely available (due to cost or hardware complications), putting dynamic cardio-vascular imaging off-limits for many researchers. Here, we propose a method that allows imaging the beating heart on conventional LSMs. Our approach takes a set of images containing scanning aberrations, each triggered at an arbitrary time in the cardiac cycle, and assembles an image sequence that covers a single cardiac heartbeat. The steps are: (i) frame sorting by solving a traveling salesman problem; (ii) heartbeat duration estimation; and (iii) scan-delay compensation via space-time resampling. We characterize the performance of our method on synthetic data under several light intensities and scanning speeds. We further illustrate our method's applicability on experimental images acquired in live zebrafish larvae, and show that the reconstruction quality approaches that of fast, state-of-the-art microscopes. Our technique opens the possibility of using LSMs to carry out studies of cardiac dynamics, without the need for prospective gating or fast microscopes.

In vivo microscopy is an important tool to study developing organs such as the heart of the zebrafish embryo but is often limited by slow image frame acquisition speed. While collections of still images of the beating heart at arbitrary phases can be sorted to obtain a virtual heartbeat, the presence of identical heart configurations at two or more heartbeat phases can derail this approach. Here, we propose a dual illumination method to encode movement in alternate frames to disambiguate heartbeat phases in the still frames. We propose to alternately acquire images with a ramp and pulse illumination then sort all successive image pairs based on the ramp-illuminated data but use the pulse-illwninated images for display and analysis. We characterized our method on synthetic data, and show its applicability on experimental data and found that an exposure time of about 7% of the heartbeat or more is necessary to encode the movement reliably in a single heartbeat with a single redundant node. Our method opens the possibility to use sorting algorithms without prior information on the phase, even when the movement presents redundant frames.

Optical microscopy is an essential tool in biology and medicine. Imaging thin, yet non-flat objects in a single shot (without relying on more sophisticated sectioning setups) remains challenging as the shallow depth of field that comes with high-resolution microscopes leads to unsharp image regions and makes depth localization and quantitative image interpretation difficult. Here, we present a method that improves the resolution of light microscopy images of such objects by locally estimating image distortion while jointly estimating object distance to the focal plane. Specifically, we estimate the parameters of a spatially-variant Point-Spread function (PSF) model using a Convolutional Neural Network (CNN), which does not require instrument- or object-specific calibration. Our method recovers PSF parameters from the image itself with up to a squared Pearson correlation coefficient of 0.99 in ideal conditions, while remaining robust to object rotation, illumination variations, or photon noise. When the recovered PSFs are used with a spatially-variant and regularized Richardson-Lucy deconvolution algorithm, we observed up to 2.1 dB better signal-to-noise ratio compared to other blind deconvolution techniques. Following microscope-specific calibration, we further demonstrate that the recovered PSF model parameters permit estimating surface depth with a precision of 2 micrometers and over an extended range when using engineered PSFs. Our method opens up multiple possibilities for enhancing images of non-flat objects with minimal need for a priori knowledge about the optical setup., Preprint

In microscopy, the time burden and cost of acquiring and annotating large datasets that many deep learning models take as a prerequisite, often appears to make these methods impractical. Can this requirement for annotated data be relaxed? Is it possible to borrow the knowledge gathered from datasets in other application fields and leverage it for microscopy? Here, we aim to provide an overview of methods that have recently emerged to successfully train learning-based methods in bio-microscopy., Accepted in Photoniques 104

Significance: Despite recent developments in microscopy, temporal aliasing can arise when imaging dynamic samples. Modern sampling frameworks, such as generalized sampling, mitigate aliasing but require measurement of temporally overlapping and potentially negative-valued inner products. Conventional cameras cannot collect these directly as they operate sequentially and are only sensitive to light intensity., Aim: We aim to mitigate aliasing in microscopy of dynamic monochrome samples by implementing generalized sampling via the use of a color camera and modulated color illumination., Approach: We solve the overlap problem by spectrally multiplexing the acquisitions and using (positive) B-spline segments as projection kernels. Reconstruction involves spectral unmixing and inverse filtering. We implemented this method using a color LED illuminator. We evaluated its performance by imaging a rotating grid and its applicability by imaging the beating zebrafish embryo heart in transmission and light-sheet microscopes., Results: Compared to stroboscopic imaging, our method mitigates aliasing with performance improving as the projection order increases. The approach can be implemented in conventional microscopes but is limited by the number of available LED colors and camera channels., Conclusions: Generalized sampling can be implemented via color modulation in microscopy to mitigate temporal aliasing. The simple hardware requirements could make it applicable to other optical imaging modalities. (C) The Authors.

Generalized sampling is a flexible framework for signal acquisition, which relaxes the need for ideal pre-filters. Nevertheless, implementation remains challenging for dynamic imaging applications because it requires simultaneously measuring multiple overlapping inner-products and because only positive signals (intensities) can be measured by cameras. We present a method to collect videos of monochromatic objects by projecting the incoming signal at each pixel in a temporal B-spline space of degree 0, 1, or 2 by using a conventional RGB camera and a modulated three-color light source for illumination. Specifically, we solve the basis function overlap problem by multiplexing the acquisition in different color ranges and use B-spline pieces (which are positive) as projection kernels of a biorthogonal projection-expansion bases pair. The steps to recover signal samples include spectral unmixing and inverse filtering. Reconstructions we obtained from simulated and experimentally-acquired microscopy data demonstrate the feasibility of our approach.

Current methods to build dynamic optical coherence tomography (OCT) volumes of the beating embryonic heart involve synchronization of 2D+time slice-sequences acquired over separate heartbeats. Temporal registration of these sequences is performed either through gating or postprocessing. While synchronization algorithms that exclusively rely on image- intrinsic signals allow forgoing external gating hardware, they are prone to error accumulation, require operator-supervised correction, or lead to nonisotropic resolution. Here, we propose an image-based, retrospective reconstruction technique that uses two sets of parallel 2D+T slice-sequences, acquired perpendicularly to each other, to yield accurate and automatic reconstructions with isotropic resolution. The method utilizes the similarity of the data at the slice intersections to spatio-temporally register the two sets of slice sequences and fuse them into a high-resolution 4D volume. We characterize our method by using 1) simulated heart phantom datasets and 2) OCT datasets acquired from the beating heart of live cultured E9.5 mouse and E10.5 rat embryos. We demonstrate that while our method requires greater acquisition and reconstruction time compared to methods that use slices from a single direction, it produces more accurate and self-validating reconstructions since each set of reconstructed slices acts as a reference for the slices in the perpendicular set.

The cardiac conduction system (CCS) propagates and coordinates the electrical excitation that originates from the pacemaker cells, throughout the heart, resulting in rhythmic heartbeat. Its defects result in life-threatening arrhythmias and sudden cardiac death. Understanding of the factors involved in the formation and function of the CCS remains incomplete. By transposon assisted transgenesis, we have developed enhancer trap (ET) lines of zebrafish that express fluorescent protein in the pacemaker cells at the sino-atrial node (SAN) and the atrio-ventricular region (AVR), termed CCS transgenics. This expression pattern begins at the stage when the heart undergoes looping morphogenesis at 36 h post fertilization (hpf) and is maintained into adulthood. Using the CCS transgenics, we investigated the effects of perturbation of cardiac function, as simulated by either the absence of endothelium or hemodynamic stimulation, on the cardiac conduction cells, which resulted in abnormal compaction of the SAN. To uncover the identity of the gene represented by the EGFP expression in the CCS transgenics, we mapped the transposon integration sites on the zebrafish genome to positions in close proximity to the gene encoding fibroblast growth homologous factor 2a (fhf2a). Fhf2a is represented by three transcripts, one of which is expressed in the developing heart. These transgenics are useful tools for studies of development of the CCS and cardiac disease.

Capturing the dynamics of individual structures in the embryonic heart is an essential step for studying its function and development. Label-free brightfield microscopy allows for higher acquisition frame-rates than techniques requiring molecular labeling, without interfering with embryo viability or needing complex equipment. However, since different structures contribute similarly to image contrast, label-free microscopy lacks specificity. Here, we mitigate this problem by separating a single-channel image series into multiple channels specific to different cardio-vascular structures, based only on their motion patterns. The technique combines images from multiple cardiac cycles and z-sections after nonuniform temporal registration to produce 3-D+time image volumes of one full cardiac cycle with separate channels for static, transient and periodically moving structures. The resulting data is well suited for velocity analysis and 3-D-visualization. We characterize the separating capabilities of our technique on a synthetic cardiac dataset and demonstrate its practical applicability, by reconstructing three-channel views of the beating embryonic zebrafish heart with an effective frame rate of 1000 volumes (256 × 256 × 20 voxels each) per second. This technique enables quantitative characterization of dynamic heart function during cardiogenesis.

Images of multiply labeled fluorescent samples provide unique insights into the localization of molecules, cells, and tissues. The ability to image multiple channels simultaneously at high speed without cross talk is limited to a few colors and requires dedicated multichannel or multispectral detection procedures. Simpler microscopes, in which each color is imaged sequentially, produce a much lower frame rate. Here, we describe a technique to image, at high frame rate, multiply labeled samples that have a repeating motion. We capture images in a single channel at a time over one full occurrence of the motion then repeat acquisition for other channels over subsequent occurrences. We finally build a high-speed multichannel image sequence by combining the images after applying a normalized mutual information-based time registration procedure. We show that this technique is amenable to image the beating heart of a double-labeled embryonic quail in three channels (brightfield, yellow, and mCherry fluorescent proteins) using a fluorescence wide-field microscope equipped with a single monochrome camera and without fast channel switching optics. We experimentally evaluate the accuracy of our method on image series from a two-channel confocal microscope.