Search

The full characterization of ultrashort pulses is crucial for all their applications. Lately, the dispersion scan (d-scan) technique has been developed, which stands out by its simplicity [1, 2]. Therefore, it is a promising candidate for a low-maintenance pulse characterization technique. The reconstruction of the spectral phase from a d-scan measurement relies on optimization algorithms [3, 4]. Considering the fact that the spectral phase of the pulses is directly imprinted on the d-scan trace, the phase reconstruction can also be regarded as an image recognition task. During the last few years, artificial neural networks have shown excellent performance in different fileds, e.g. image recognition [5].

This document describes two reference video traffic models for evaluating RTP congestion control algorithms. The first model statistically characterizes the behavior of a live video encoder in response to changing requests on the target video rate. The second model is trace-driven and emulates the output of actual encoded video frame sizes from a high-resolution test sequence. Both models are designed to strike a balance between simplicity, repeatability, and authenticity in modeling the interactions between a live video traffic source and the congestion control module. Finally, the document describes how both approaches can be combined into a hybrid model.

In this paper, a quality-driven packet scheduling algorithm for DASH streaming is proposed for the LTE Wi-Fi Aggregation (LWA) systems. The playback buffer status on the receiver side and the network conditions for LTE and Wi-Fi are used to effectively control the video rate. To estimate the network conditions, the distributions of the packet inter-arrival times of both paths are used. The playback buffer is modeled to find the probability of the successful arrival of the assigned video packets before the playback buffer depletes. A convex optimization method is used to find the optimal packet distribution for the LTE and Wi-Fi. Simulation results based on NS-3 show that the proposed algorithm can stream the video with the best quality in terms of the utility while having less probability than other methods of re-buffering while the video is playing.

Long-wave infrared imaging systems remain prohibitively expensive for many high-pixel-count applications, such as panoramic imaging. We present a computational imaging solution which allows for low-cost, 360° long-wave infrared panoramic imaging at video rate.

Graphical Processing Units (GPUs) are a powerful alternative to central processing units, especially for data-parallel, video-rate processing of large data volumes. In the complex scenario of high-performance, multichannel Time Correlated Single Photon Counting (TCSPC), a huge amount of data is potentially generated by the acquisition system. Exploiting a dedicated, external, programmable elaboration unit enables a high degree of flexibility to perform different types of analysis. In this paper, we present a GPU-based application that leverages the common unified device architecture application programming interface for video-rate and accurate lifetime extraction from TCSPC data acquired at a rate of up to 10 Gbit/s.Graphical Processing Units (GPUs) are a powerful alternative to central processing units, especially for data-parallel, video-rate processing of large data volumes. In the complex scenario of high-performance, multichannel Time Correlated Single Photon Counting (TCSPC), a huge amount of data is potentially generated by the acquisition system. Exploiting a dedicated, external, programmable elaboration unit enables a high degree of flexibility to perform different types of analysis. In this paper, we present a GPU-based application that leverages the common unified device architecture application programming interface for video-rate and accurate lifetime extraction from TCSPC data acquired at a rate of up to 10 Gbit/s.

High-speed atomic force microscopy (HS-AFM) has enabled observing protein molecules during their functional activity at rates of 1-12.5 frames per second (fps), depending on the imaging conditions, sample height, and fragility. To meet the increasing demand for the great expansion of observable dynamic molecular processes, faster HS-AFM with less disturbance is imperatively needed. However, even a 50% improvement in the speed performance imposes tremendous challenges, as the optimization of major rate-limiting components for their fast response is nearly matured. This paper proposes an alternative method that can lower the feedback control error and thereby enhance the imaging rate. This method can be implemented in any HS-AFM system by minor modifications of the software and hardware. The resulting faster and less-disturbing imaging capabilities are demonstrated by the imaging of relatively fragile actin filaments and microtubules near the video rate, and of actin polymerization that occurs through weak intermolecular interactions, at ∼8 fps.

To date, numerous studies have been performed to elucidate the complex cellular dynamics in skin diseases, but few have attempted to characterize these cellular events under conditions similar to the native environment. To address this challenge, a three-dimensional (3D) multimodal analysis platform was developed for characterizing in vivo cellular dynamics in skin, which was then utilized to process in vivo wound healing data to demonstrate its applicability. Special attention is focused on in vivo biological parameters that are difficult to study with ex vivo analysis, including 3D cell tracking and techniques to connect biological information obtained from different imaging modalities. These results here open new possibilities for evaluating 3D cellular dynamics in vivo, and can potentially provide new tools for characterizing the skin microenvironment and pathologies in the future.

Terahertz (THz) imaging has been demonstrated in numerous applications from medical to non-destructive evaluation (NDE), but current systems require expensive components, provide slow frame-rates and low resolutions. THz holography offers a potentially low-cost, high-performance alternative. Here we demonstrate the first full video-rate THz digital holography system at 2.52 THz (118.8 µm) using low-cost optical components. 2D digital reconstructions of samples are performed at frame-rates of 50 Hz - an order of magnitude higher than previous systems, whilst imaging of samples concealed in common packaging types demonstrates suitability for NDE applications. A lateral resolution of 250 µm was determined using a 1951 USAF target.

In this paper, we present a portable, video rate, low-cost and high imaging quality terahertz imager which is suitable for manned spacecraft on-orbit nondestructive examination and security inspection, the imager can hand-held by astronaut or space station's robot arm, and some components test results are given.

Solid‐state reflective displays (SRD®) have been proposed as a new route for next generation reflective displays. We present the first optical measurements of a combined RGBW states, together with a simulated black state. These results demonstrate the feasibility of a future high performance, video capable, full color, solid‐state reflective display.

A mobile testbed capable of real-time (10 Hz) standoff microwave imaging for concealed threat detection is presented. The system samples a scene with a 1m x 1m multistatic switched antenna array operating at 24–29 GHz. It constructs images on a cost-effective processor at video rate, through the use of Fast Fourier Transform (FFT) imaging. Design and operation of the testbed are discussed, and representative experimental images are presented.

We use plasmon rulers to follow the conformational dynamics of a single protein for up to 24 h at a video rate. The plasmon ruler consists of two gold nanospheres connected by a single protein linker. In our experiment, we follow the dynamics of the molecular chaperone heat shock protein 90 (Hsp90), which is known to show “open” and “closed” conformations. Our measurements confirm the previously known conformational dynamics with transition times in the second to minute time scale and reveals new dynamics on the time scale of minutes to hours. Plasmon rulers thus extend the observation bandwidth 3–4 orders of magnitude with respect to single-molecule fluorescence resonance energy transfer and enable the study of molecular dynamics with unprecedented precision.

Video rate adaptation has large impact on quality of experience (QoE). However, existing video rate adaptation is rather limited due to a small number of rate choices, which results in (i) under-selection, (ii) rate fluctuation, and (iii) frequent rebuffering. Moreover, selecting a single video rate for a 360° video can be even more limiting, since not all portions of a video frame are equally important. To address these limitations, we identify new dimensions to adapt user QoE - dropping video frames, slowing down video play rate, and adapting different portions in 360° videos. These new dimensions along with rate adaptation give us a more fine-grained adaptation and significantly improve user QoE. We further develop a simple yet effective learning strategy to automatically adapt the buffer reservation to avoid performance degradation beyond optimization horizon. We implement our approach Favor in VLC, a well known open source media player, and demonstrate that Favor on average out-performs Model Predictive Control (MPC), rate-based, and buffer-based adaptation for regular videos by 24%, 36%, and 41%, respectively, and 2X for 360° videos.

Stimulated Raman scattering (SRS) is non-linear vibrational spectroscopic technique with sub-millisecond sampling times allowing to perform video rate imaging on biological objects including even few tens of cells simultaneously. In this work SRS was used for in vivo study of cell division in zebrafish embryo in order to get insight into temporal evolution of the related processes.

We present a single-shot real-time video-rate speckle reduction technique for optical coherence tomography (OCT) using an adaptation of the dual window (DW) method. This computational technique requires no hardware modification and offers approximately 25% reduction in speckle contrast without reducing spatial resolution.

We present a new type of confocal microscope that simultaneously image a specimen at four different depths, thus providing volumetric imaging at video rate. Our technique is an attractive tool for in vivo high-speed volumetric calcium imaging.

© 2018 The Author(s). A holography computing board that combines eight field-programmable gate arrays offers a scalable approach to generating holograms with a high frame rate.

We present a graphics processing unit (GPU)-accelerated optical coherence elastography (OCE) system capable of generating strain images (elastograms) of soft tissue at near video-rates. The system implements phase-sensitive compression OCE using a pipeline of GPU kernel functions to enable a highly parallel implementation of OCE processing using the OpenCL framework. Developed on a commercial-grade GPU and desktop computer, the system achieves a processing rate of 21 elastograms per second at an image size of 960 × 400 pixels, enabling high-rate visualization during acquisition. The system is demonstrated on both tissue-simulating phantoms and fresh ex vivo mouse muscle. To the best of our knowledge, this is the first implementation of near video-rate OCE and the fastest reported OCE processing rate, enabling, for the first time, a system capable of computing and displaying OCE elastograms interactively during acquisition. This advance provides new opportunities for medical imaging of soft tissue stiffness using optical methods.

A key hurdle to achieve video-rate atomic force microscopy (AFM) in constant-force contact mode is the inadequate bandwidth of the vertical feedback control loop. This paper describes techniques used to increase the vertical tracking bandwidth of a nanopositioner to a level that is sufficient for video-rate AFM. These techniques involve the combination of: a high-speed XYZ nanopositioner; a passive damping technique that cancels the inertial forces of the Z actuator which in turns eliminates the low 20-kHz vertical resonant mode of the nanopositioner; an active control technique that is used to augment damping to high vertical resonant modes at 60 kHz and above. The implementation of these techniques allows a tenfold increase in the vertical tracking bandwidth, from 2.3 (without damping) to 28.1 kHz. This allows high-quality, video-rate AFM images to be captured at 10 frames/s without noticeable artifacts associated with vibrations and insufficient vertical tracking bandwidth.

We present a method of time coding with ABAB synchronization timing control for real-time 3D super-resolution range-gated imaging (3DSRGI). To meet the high precision of time delay and pulse width in ABAB synchronization time sequencing, phase shift is implemented to achieve ns-scaled delay and width accuracy without restoring to high clock frequencies. Theoretical analysis and experiments prove that 1 ns delay and width precision is obtained by our timing control unit based on a single field-programmable gate array with 5 ns clock cycle. Finally, a prototype experiment of 3DSRGI is demonstrated at a 10 Hz video rate with 696 pixels×520 pixels.

3D images of moving objects can be achieved with a surveillance camera and four white light-emitting diodes. With these simple components, an imaging rate of 15 Hz is possible, limited by the camera framerate.

Aiming at the problems of the DASH technology in the initial delay of video playback, the number and duration time which the video is buffered again and the switching frequency between different quality video, an improved QoE-based rate-adaptive switching algorithm is proposed. The algorithm reduces the initial delay of the video by using the lowest bit rate video clip as the initial play file. By designing a fast start algorithm, it can improve the video rate as soon as the current bandwidth environment is satisfied. The algorithm converts the video into multiple rate video, so that the again video buffer phenomenon will not appear when the average bandwidth of the network is greater than the minimum rate video, and the quality of the video playback can be improved in the case of the minimum initial delay, and the average quality of the video can be improved at the lowest initial delay. In order to effectively reduce the average switching times of video rate, the algorithm determines whether to switch the video rate by judging the current buffer state and the average bandwidth over a period of time. Experimental results show that the algorithm can reduce the number of again video buffering and the switching frequency between different quality video, and improved the quality of experience (QoE) of the user to DASH service.

Light-field microscopy (LFM) is a scalable approach for volumetric Ca2+ imaging with the highest volumetric acquisition rates (up to 100 Hz). While this has enabled high-speed whole-brain Ca2+ imaging in small semi-transparent specimen, tissue scattering has limited its application in the rodent brain. Here we introduce Seeded Iterative Demixing (SID), a computational source extraction technique that extends LFM to the scattering mammalian cortex. Using GCaMP-expressing mice we demonstrate SID’s ability to capture neuronal dynamics in vivo within a volume of 900×900×260μm located as deep as 380 μm in the mouse cortex and hippocampus at 30 Hz volume rate while faithfully discriminating signals from neurons as close as 20 μm, at three orders of magnitude reduced computational cost. The simplicity and scalability of LFM, coupled with the performance of SID opens up a range of new applications including closed-loop experiments and is expected to propel its wide dissemination within the neuroscience community.

This paper reports a fully packaged confocal endomicroscope high resolution and high frame-rate (HRHF) Lissajous fiber scanning. The confocal endomicroscope features a resonant scanning fiber with ∼1kHz actuated by a piezoelectric tube (PZT). The Lissajous scanning with high resolution and high frame rate has been successfully achieved by using the selection rule of scanning frequency, i.e., strong correlation between the total lobe number of Lissajous images and the greatest common divisor (GCD) between two scanning frequencies. Our main results clearly demonstrate exceptional fill factor of 85 % at 10 Hz in frame rate. Besides, this fully packaged endomicroscopic catheter was further combined with a portable confocal microscopic system to obtain video-rate 2D reflectance as well as in-vivo mouse vascular imaging.

Fluorescence imaging is well suited to provide image guidance during resections in oncologic and vascular surgery. However, the distorting effects of tissue optical properties on the emitted fluorescence are poorly compensated for on even the most advanced fluorescence image guidance systems, leading to subjective and inaccurate estimates of tissue fluorophore concentrations. Here we present a novel fluorescence imaging technique that performs real-time (i.e., video rate) optical property corrected fluorescence imaging. We perform full field of view simultaneous imaging of tissue optical properties using Single Snapshot of Optical Properties (SSOP) and fluorescence detection. The estimated optical properties are used to correct the emitted fluorescence with a quantitative fluorescence model to provide quantitative fluorescence-Single Snapshot of Optical Properties (qF-SSOP) images with less than 5% error. The technique is rigorous, fast, and quantitative, enabling ease of integration into the surgical workflow with the potential to improve molecular guidance intraoperatively.

Two-photon fluorescence lifetime imaging microscopy (TP-FLIM) of reduced nicotinamide adenine dinucleotide is a sensitive metabolic imaging modality. Here, a video rate TP-FLIM system is demonstrated for metabolic imaging in cells and tissues.

We present an analysis and a systematic design methodology for a novel nonraster scan method based on a rosette pattern and demonstrate its application in video-rate atomic force microscopy. This pattern is traced when the lateral axes of a parallel kinematic scanner are commanded to follow a combination of two sinusoids with identical amplitudes and different frequencies. We design an internal-model-based controller to enhance the tracking performance of this pattern and implement the scheme on a microelectromechanical system scanner. The results reveal high-precision tracking of the rosette pattern in order to acquire time-lapsed atomic force microscope images at the rate of 10 frames/s.

Optoacoustic imaging provides a unique combination of high optical contrast and excellent spatial resolution, making it ideal for simultaneous imaging of tissue anatomy as well as functional and molecular contrast in deep optically opaque tissues. We report on development of a portable clinical system for three-dimensional optoacoustic visualization of deep human tissues at video rate. Studies in human volunteers have demonstrated powerful performance in delivering high resolution volumetric multispectral optoacoustic tomography (vMSOT) images of tissue morphology and function, such as blood oxygenation parameters, in real time. Whilst most imaging modalities currently in clinical use are not able to deliver volumetric data with comparable time resolution, the presented imaging approach holds promise to attain new diagnostic and treatment monitoring value for multiple indications, such as cardiovascular and peripheral vascular disease, disorders related to the lymphatic system, breast lesions, arthritis and inflammation.

Atomic Force Microscopy (AFM) is a powerful instrument for studying and exploring the nanoworld [1]. AFM can obtain ultrahigh-resolution images at the subnanoscale level. However, AFM has a very significant drawback of slow imaging speed, which is due to its working principle. A conventional AFM conducts a raster scan of an entire area to generate a topography image. Therefore, the frame rate is low, making it impossible for observation of biological and physical processes that are dynamic in nature with a lifespan of a few minutes or even seconds, such as the structural change of cells, carbon nanotube shape change, and so forth [2]?[5]. In addition, for AFM-based nanomanipulations and nanomeasurement, the low frame rate makes it difficult to achieve a real-time visual guide manipulation [6], [7]. Operators usually have to wait for finishing imaging to visualize the manipulating results. Therefore, there is an increasing demand on a fast-imaging AFM system that can capture a continuous phenomenon occurring in seconds.

In-situ aberration-corrected transmission electron microscopy combined with density functional theory (DFT) calculations are employed to elucidate the electron beam induced dynamics of surface dislocations on ~ 4 nm × 6 nm sized Au (100) terraces located on top of hillock shaped Au at room temperature and 77 K at video rate (20 frames/s). At room temperature, very small terraces (

In this article, we report our study of a dual mode reflective cholesteric display, which works in the traditional bistable mode as well as in the new and fast monostable mode. Because both modes are compatible, they can be used according to the need for video or static images. The bistable mode operates between the reflecting planar state and non-reflecting focal conic state, both of which are stable at zero voltage. The bistable mode is suitable for displaying static images. In the monostable mode, it is operated on wrinkled cholesteric layers. It is fast and has low driving voltage, and can exhibit gray levels of reflectance determined by the applied voltage. The monostable mode is suitable to display video rate dynamic images.

Body scanning presents unique value in delivering the first digital asset of a human body thus resulting a fundamental device for a range of applications dealing with health, fashion and fitness. Despite several body scanners are in the market, recently depth cameras such as Microsoft Kinect® have attracted the 3D community; compared with conventional 3D scanning systems, these sensors are able to capture depth and RGB data at video rate and even if quality and depth resolution are not optimal for this kind of applications, the major benefit comes from the overall acquisition speed and from the IR pattern that allows poor lighting conditions optimal acquisition. When dealing with non-rigid bodies, unfortunately, the use of a single depth camera may lead to inconsistent results mainly caused by wrong surfaces registration. With the aim of improving existing systems based on low-resolution depth cameras, the present paper describes a novel scanning system for capturing 3D full human body models by using multiple Kinect® devices in a compact setup. The system consists of an instantaneous scanning system using eight depth cameras, appropriately arranged in a compact wireframe. To validate the effectiveness of the proposed architecture, a comparison of the obtained 3D body model with the one obtained using a professional Konica Minolta Range Seven 3D scanner is also presented and possible drawbacks are hinted at.

Using simultaneous localization and mapping to determine the 3D surroundings and pose of a wearable or hand-held camera provides the geometrical foundation for several capabilities of value to an autonomous wearable vision system. The one explored here is the ability to incorporate recognized objects into t he map of the surroundings and refer to them. Established methods for feature cluster r ecognition are used to identify and localize known planar objects, and their geometry is incorporated into the map of the surrounds using a minimalist representation. Continued measurement of these mapped objects improves both the accuracy of estimated maps and the robustness of the tracking system. In the context of wearable (or hand-held) vision, the system’s ability to enhance generated maps with known objec ts increases the map’s value to human operators, and also enables meaningful automatic annotation of the user’s surroundings.

Neurons and neural networks often extend hundreds to thousands of micrometers in three dimensions. To capture all the calcium transients associated with their activity, we need volume imaging methods with sub-second temporal resolution. Such speed is challenging for conventional two-photon laser scanning microscopy (2PLSM) to achieve, because of its dependence on serial focal scanning in 3D and the limited brightness of indicators. Here we present an optical module that can be easily integrated into standard 2PLSMs to generate an axially elongated Bessel focus. Scanning the Bessel focus in 2D turned frame rate into volume rate and enabled video-rate volumetric imaging. Using Bessel foci designed to maintain synaptic-level lateral resolution in vivo, we demonstrated the power of this approach in enabling discoveries for neurobiology by imaging the calcium dynamics of volumes of neurons and synapses in fruit flies, zebrafish larvae, mice, and ferrets in vivo.

Free Viewpoint Video and TV are technologies regarded as the future of digital entertainment allowing users to navigate through multiple video streams of an event to select novel viewpoints. In this paper, we propose an automatic real-time video stitching algorithm that is capable of producing spherical panoramas for FTV applications. The algorithm works by splitting automatic image stitching into one-time and recurring computations. We utilize GPUs for accelerating the recurring computations and achieving real-time performance. By effectively caching the repeated transform maps, we improve the stitching performance by two orders of magnitude compare to CPU implementations. The multiple stitching steps are pipelined to distribute the panorama stitching computations between multiple processing nodes to produce visually appealing panoramas at 30Hz without any noticeable blurring or stitching artifacts for five HD synchronized cameras.

The design of an adaptive video client for mobile users is challenged by the frequent changes in operating conditions. Such conditions present a seemingly insurmountable challenge to adaptation algorithms, which may fail to find a balance between video rate, stalls, and rate-switching. In an effort to achieve the ideal balance, we design OSCAR, a novel adaptive streaming algorithm whose adaptation decisions are optimized to avoid stalls while maintaining high video quality. Our performance evaluation, using real video and channel traces from both 3G and 4G networks, shows that OSCAR achieves the highest percentage of stall-free sessions while maintaining a high quality video in comparison to the state-of-the-art algorithms.

We implement Poisson denoising algorithms that are fast enough to allow real-time processing of images produced by a single photon avalanche detector (SPAD) camera. This work has applications in low-light-level surveillance and medical imaging.

We demonstrate the monolithic integration of an active photo-pixel made in InSb on a GaAs substrate. Video rate sampling of photo-generated signal can be provided by the co-integration of a GaAs MESFET with an InSb photodiode.

We developed a dual mode reflective cholesteric display which works in the traditional bistable mode as well as in a monostable mode. In the bistable mode, it is operated between the reflecting planar state and non-reflecting focal conic state, both of which are stable at zero voltage. The bistable mode is suitable to display static images. In the monostable mode, it is operated on all the intermediate states between the planar and focal conic states. It is fast and has low driving voltage, and can exhibit gray levels of reflectance determined by the applied voltage. The monostable mode is suitable to display video rate dynamic images.

Electronic readers that use rugged flexible substrates potentially offer very attractive characteristics (low-cost processing, lightweight, mechanical flexibility, etc.) and could expand the explosive growth of e-reader devices. In this paper, we demonstrate electrowetting (EW) devices fabricated on flexible substrates, including paper, polymers (plastics), and metal foils and sheets. EW devices on some of the flexible substrates exhibit characteristics very close to those of conventional EW devices on glass substrates. Prototypes of flexible EW arrays on plastic substrates are demonstrated to switch reversibly by applying a low voltage difference (20 V). The array operation is maintained even when the display is mechanically flexed. These results indicate the promise of flexible EW devices for mobile and other devices, including video rate flexible e-paper.

Many disease states are associated with cellular biomechanical changes as markers. Label-free phase microscopes are used to quantify thermally driven interface fluctuations, which allow the deduction of important cellular rheological properties. Here, the spatio-temporal coherence of light was used to implement a high-speed reflection phase microscope with superior depth selectivity and higher phase sensitivity. Nanometric scale motion of cytoplasmic structures can be visualized with fine details and three-dimensional resolution. Specifically, the spontaneous fluctuation occurring on the nuclear membrane of a living cell was observed at video rate. By converting the reflection phase into displacement, the sensitivity in quantifying nuclear membrane fluctuation was found to be about one nanometer. A reflection phase microscope can potentially elucidate biomechanical mechanisms of pathological and physiological processes.

We report the first computational super-resolved, multi-camera integral imaging at long-wave infrared (LWIR) wavelengths. A synchronized array of FLIR Lepton cameras was assembled, and computational super-resolution and integral-imaging reconstruction employed to generate video with light-field imaging capabilities, such as 3D imaging and recognition of partially obscured objects, while also providing a four-fold increase in effective pixel count. This approach to high-resolution imaging enables a fundamental reduction in the track length and volume of an imaging system, while also enabling use of low-cost lens materials., Supplementary multimedia material in http://dx.doi.org/10.6084/m9.figshare.5303020