Search

Supplemental Video 2: Endoscopic high-speed videos - precancerous lesions. The video shows a short sequence of the laryngeal high-speed video from each subject of the precancerous lesion group. The segmentation of the vocal folds is also shown (red: right vocal fold, blue: left vocal fold)

Supplemental PVG Figure 2. The phonovibrogram (PVG) from each subject of the precancerous lesion group is shown. The PVG is explained more detailed within the manuscript.

Supplemental PVG Figure 1. The phonovibrogram (PVG) from each subject of the control group is shown. The PVG is explained more detailed within the manuscript.

Supplemental PVG Figure 3. The phonovibrogram (PVG) from each subject of the cancer group is shown. The PVG is explained more detailed within the manuscript.

Supplemental Video 3: Endoscopic high-speed videos - T1a carcinoma. The video shows a short sequence of the laryngeal high-speed video from each subject of the cancer group. The segmentation of the vocal folds is also shown (red: right vocal fold, blue: left vocal fold)

Supplemental Video 1: Endoscopic high-speed videos - control group. The video shows a short sequence of the laryngeal high-speed video from each subject of the control group. The segmentation of the vocal folds is also shown (red: right vocal fold, blue: left vocal fold)

Objectives This study aimed to systematically investigate whether plaque autofluorescence properties assessed with intravascular fluorescence lifetime imaging (FLIm) can provide qualitative and quantitative information about intimal composition and improve the characterization of atherosclerosis lesions. Background Despite advances in cardiovascular diagnostics, the analytic tools and imaging technologies currently available have limited capabilities for evaluating in situ biochemical changes associated with luminal surface features. Earlier studies of small number of samples have shown differences among the autofluorescence lifetime signature of well-defined lesions, but a systematic pixel-level evaluation of fluorescence signatures associated with various histological features is lacking and needed to better understand the origins of fluorescence contrast. Methods Human coronary artery segments (n = 32) were analyzed with a bimodal catheter system combining multispectral FLIm with intravascular ultrasonography compatible with in vivo coronary imaging. Various histological components present along the luminal surface (200-μm depth) were systematically tabulated (12 sectors) from each serial histological section (n = 204). Morphological information provided by ultrasonography allowed for the accurate registration of imaging data with histology data. The relationships between histological findings and FLIm parameters obtained from 3 spectral channels at each measurement location (n = 33,980) were characterized. Results Our findings indicate that fluorescence lifetime from different spectral bands can be used to quantitatively predict the superficial presence of macrophage foam cells (mFCs) (area under the receiver-operator characteristic curve: 0.94) and extracellular lipid content in advanced lesions (lifetime increase in 540-nm band), detect superficial calcium (lifetime decrease in 450-nm band area under the receiver-operator characteristic curve: 0.90), and possibly detect lesions consistent with active plaque formation such as pathological intimal thickening and healed thrombus regions (lifetime increase in 390-nm band). Conclusions Our findings indicate that autofluorescence lifetime provides valuable information for characterizing atherosclerotic lesions in coronary arteries. Specifically, FLIm can be used to identify key phenomena linked with plaque progression (e.g., peroxidized-lipid–rich mFC accumulation and recent plaque formation).

District Heating Systems (DHS) distribute heat in terms of hot water or steam. Loss of media (water or steam), and thus energy, is expensive and has a negative impact on the environment. It is therefore of great interest to develop techniques to detect and localize potential leakages fast and cost-effectively. To avoid interference with the operating process of a DHS, airborne thermography comes into place. The use of Unmanned Aerial Vehicles (UAV) as a flexible and low-cost platform equipped with a Thermal InfraRed (TIR) camera is a promising alternative to a conventional manned flight.This paper describes a method for automatic thermal anomaly detection of a DHS. Thermal data acquisition using a UAV is followed by photogrammetric processing of the TIR images. In this way, a thermal orthophoto is produced. The next step is an identification of anomalies by means of image analysis. We apply the Laplacian of Gaussian (LoG) blob detector to find high temperature regions in areas of interest of a thermal orthomosaic. This area of interest is defined around the DHS position in the images as defined in a Geographic Information System. Finally, segmentation and classification are employed to reduce false alarms and localize thermal anomalies. An experimental evaluation using real-world data is presented, showing that the developed method deliverers promising results.

Robotic and endoscopic surgery is increasingly used in clinical practice and typically relies on stereoscopic vision to enable 3D visualization of the surgical field. We combined this capability with a FLIm acquisition system suitable for the identification of tumor tissue to generate a 3D map of the surgical field that comprises both FLIm and white light image information. This result is achieved using semi-global matching and deep stereo matching neural network. In addition to the generation of a 3D model of the surgical cavity, this approach leads to a more realistic rendering of FLIm maps by including tissue shading.

Real-time visualization of imaging data constitutes a critical part of surgical workflow. Augmented reality (AR) is a promising tool to assist in conventional surgical navigation systems. We have been developing an AR framework for clinical imaging and guidance using an optical see-through head-mounted display (OST-HMD) and fluorescence lifetime imaging (FLIm) instrumentation. This framework supports in vivo scanning of FLIm data and the real-time visualization of diagnostic information overlaid on the interrogated tissue area. With the high discriminative power of FLIm, our FLIm-AR concept has the potential for indicating tumor margins and assisting with tumor excision surgery.

Tumor-free surgical margins are critical in breast-conserving surgery. In up to 38% of the cases, however, patients undergo a second surgery since malignant cells are found at the margins of the excised resection specimen. Thus, advanced imaging tools are needed to ensure clear margins at the time of surgery. The objective of this study was to evaluate a random forest classifier that makes use of parameters derived from point-scanning label-free fluorescence lifetime imaging (FLIm) measurements of breast specimens as a means to diagnose tumor at the resection margins and to enable an intuitive visualization of a probabilistic classifier on tissue specimen. FLIm data from fresh lumpectomy and mastectomy specimens from 18 patients were used in this study. The supervised training was based on a previously developed registration technique between autofluorescence imaging data and cross-sectional histology slides. A pathologist’s histology annotations provide the ground truth to distinguish between adipose, fibrous, and tumor tissue. Current results demonstrate the ability of this approach to classify the tumor with 89% sensitivity and 93% specificity and to rapidly (∼ 20 frames per second) overlay the probabilistic classifier overlaid on excised breast specimens using an intuitive color scheme. Furthermore, we show an iterative imaging refinement that allows surgeons to switch between rapid scans with a customized, low spatial resolution to quickly cover the specimen and slower scans with enhanced resolution (400 μm per point measurement) in suspicious regions where more details are required. In summary, this technique provides high diagnostic prediction accuracy, rapid acquisition, adaptive resolution, nondestructive probing, and facile interpretation of images, thus holding potential for clinical breast imaging based on label-free FLIm.

Tumor removal typically involves electrocautery, but no studies to date have quantified the effect of electrocautery on fluorescence emission. Electrocautery was applied to N=4 locations of the oral cavity and striated leg muscle of a live Yorkshire pig. Autofluorescence of cauterized tissues and surrounding regions was measured at distinct time points up to 120 minutes following cauterization. The fluorescence lifetime was spectrally resolved in four spectral detection channels that maximized the signal emanating from endogenous fluorophores of interest. The autofluorescence emission (355 nm excitation) was temporally resolved using a high-speed digitizer; resulting fluorescence decay characteristics were retrieved using the Laguerre deconvolution technique. Histology was performed and co-registered with the autofluorescence data. Results show that cauterized tissue presents a distinct autofluorescence signature from surrounding regions immediately after cauterization. Differences become less evident with time. The autofluorescence-derived parameters suggest altered metabolism in peripheral regions compared to the region of maximal damage. Within the time-frame of this study, tissues investigated show variable degrees of recovery from the effects of electrocautery that can be monitored by changes in fluorescence lifetime characteristics. Our findings suggest delineation of pathologic conditions could be affected by tissue cauterization and that future studies in this area will be necessary.

Current clinical brain imaging techniques used for surgical planning of tumor resection lack intraoperative and real-time feedback; hence surgeons ultimately rely on subjective evaluation to identify tumor areas and margins. We report a fluorescence lifetime imaging (FLIm) instrument (excitation: 355 nm; emission spectral bands: 390/40 nm, 470/28 nm, 542/50 nm, and 629/53 nm) that integrates with surgical microscopes to provide real-time intraoperative augmentation of the surgical field of view with fluorescent derived parameters encoding diagnostic information. We show the functionality and safety features of this instrument during neurosurgical procedures in patients undergoing craniotomy for the resection of brain tumors and/or tissue with radiation damage. We demonstrate in three case studies the ability of this instrument to resolve distinct tissue types and pathology including cortex, white matter, tumor, and radiation-induced necrosis. In particular, two patients with effects of radiation induced necrosis exhibited longer fluorescence lifetimes and increased optical redox ratio on the necrotic tissue with respect to non-affected cortex, and an oligodendroglioma resected from a third patient reported shorter fluorescence life-time and a decrease in optical redox ratio than the surrounding white matter. These results encourage the use of FLIm as a label-free and non-invasive intraoperative tool for neurosurgical guidance.

In engineering geodesy, the technical progress leads to various kinds of multi-sensor systems (MSS) capturing the environment. Multi-sensor systems, especially those mounted on unmanned aerial vehicles, subsequently called unmanned aerial system (UAS), have emerged in the past decade. Georeferencing for MSS and UAS is an indispensable task to obtain further products of the data captured. Georeferencing comprises at least the determination of three translations and three rotations. The availability and accuracy of Global Navigation Satellite System (GNSS) receivers, inertial measurement units, or other sensors for georeferencing is not or not constantly given in urban scenarios. Therefore, we utilize UAS-based laser scanner measurements on building facades. The building latter are modeled as planes in a three-dimensional city model. We determine the trajectory of the UAS by combining the laser scanner measurements with the plane parameters. The resulting implicit measurement equations and nonlinear equality constraints are covered within an iterated extended Kalman filter (IEKF). We developed a software simulation for testing the IEKF using different scenarios to evaluate the functionality, performance, strengths, and remaining challenges of the IEKF implemented.

In this work we combine Time-Resolved Fluorescence Spectroscopy with machine learning techniques to provide a real-time assessment of tumor positive resection margins.

Fluorescence lifetime imaging (FLIm) was integrated with a da Vinci Surgical System (Intuitive Surgical, Inc.) for exploration of head and neck cancer (N=25 patients). In vivo results showed FLIm distinguished between tumor and healthy tissue.

Re-excision rates for breast cancer lumpectomy procedures are currently nearly 25% due to surgeons relying on inaccurate or incomplete methods of evaluating specimen margins. The objective of this study was to determine if cancer could be automatically detected in breast specimens from mastectomy and lumpectomy procedures by a classification algorithm that incorporated parameters derived from fluorescence lifetime imaging (FLIm). This study generated a database of co-registered histologic sections and FLIm data from breast cancer specimens (N = 20) and a support vector machine (SVM) classification algorithm able to automatically detect cancerous, fibrous, and adipose breast tissue. Classification accuracies were greater than 97% for automated detection of cancerous, fibrous, and adipose tissue from breast cancer specimens. The classification worked equally well for specimens scanned by hand or with a mechanical stage, demonstrating that the system could be used during surgery or on excised specimens. The ability of this technique to simply discriminate between cancerous and normal breast tissue, in particular to distinguish fibrous breast tissue from tumor, which is notoriously challenging for optical techniques, leads to the conclusion that FLIm has great potential to assess breast cancer margins. Identification of positive margins before waiting for complete histologic analysis could significantly reduce breast cancer re-excision rates.

Multi-Spectral Time-Resolved Fluorescence Spectroscopy (ms-TRFS) can provide label-free real-time feedback on tissue composition and pathology during surgical procedures by resolving the fluorescence decay dynamics of the tissue. Recently, an ms-TRFS system has been developed in our group, allowing for either point-spectroscopy fluorescence lifetime measurements or dynamic raster tissue scanning by merging a 450 nm aiming beam with the pulsed fluorescence excitation light in a single fiber collection. In order to facilitate an augmented real-time display of fluorescence decay parameters, the lifetime values are back projected to the white light video. The goal of this study is to develop a 3D real-time surface reconstruction aiming for a comprehensive visualization of the decay parameters and providing an enhanced navigation for the surgeon. Using a stereo camera setup, we use a combination of image feature matching and aiming beam stereo segmentation to establish a 3D surface model of the decay parameters. After camera calibration, texture-related features are extracted for both camera images and matched providing a rough estimation of the surface. During the raster scanning, the rough estimation is successively refined in real-time by tracking the aiming beam positions using an advanced segmentation algorithm. The method is evaluated for excised breast tissue specimens showing a high precision and running in real-time with approximately 20 frames per second. The proposed method shows promising potential for intraoperative navigation, i.e. tumor margin assessment. Furthermore, it provides the basis for registering the fluorescence lifetime maps to the tissue surface adapting it to possible tissue deformations.

The current standard of care for early stages of breast cancer is breast-conserving surgery (BCS). BCS involves a lumpectomy procedure, during which the tumor is removed with a rim of normal tissue-if cancer cells found in that rim of tissue, it is called a positive margin and means part of the tumor remains in the breast. Currently there is no method to determine if cancer cells exist at the margins of lumpectomy specimens aside from time-intensive histology methods that result in reoperations in up to 38% of cases. We used fluorescence lifetime imaging (FLIm) to measure time-resolved autofluorescence from N=13 ex vivo human breast cancer specimens (N=10 patients undergoing lumpectomy or mastectomy) and compared our results to histology. Tumor (both invasive and ductal carcinoma in situ), fibrous tissue, fat and fat necrosis have unique fluorescence signatures. For instance, between 500-580 nm, fluorescence lifetime of tumor was shortest (4.7 ± 0.4 ns) compared to fibrous tissue (5.5 ± 0.7 ns) and fat (7.0 ± 0.1 ns), P

In order to objectively assess the laryngeal vibratory behavior, endoscopic high-speed cameras capture several thousand frames per second of the vocal folds during phonation. However, judging all inherent clinically relevant features is a challenging task and requires well-founded expert knowledge. In this study, an automated wavelet-based analysis of laryngeal high-speed videos based on phonovibrograms is presented. The phonovibrogram is an image representation of the spatiotemporal pattern of vocal fold vibration and constitutes the basis for a computer-based analysis of laryngeal dynamics. The features extracted from the wavelet transform are shown to be closely related to a basic set of video-based measurements categorized by the European Laryngological Society for a subjective assessment of pathologic voices. The wavelet-based analysis further offers information about irregularity and lateral asymmetry and asynchrony. It is demonstrated in healthy and pathologic subjects as well as for a surgical group that was examined before and after the removal of a vocal fold polyp. The features were found to not only classify glottal closure characteristics but also quantify the impact of pathologies on the vibratory behavior. The interpretability and the discriminative power of the proposed feature set show promising relevance for a computer-assisted diagnosis and classification of voice disorders.

BMC evolutionary biology 16, 248 (2016). doi:10.1186/s12862-016-0827-5, Published by BioMed Central, London

Summary Introduction In a recent publication, the phasegram, a bifurcation diagram over time, has been introduced as an intuitive visualization tool for assessing the vibratory states of oscillating systems. Here, this nonlinear dynamics approach is augmented with quantitative analysis parameters, and it is applied to clinical laryngeal high-speed video (HSV) endoscopic recordings of healthy and pathological phonations. Methods HSV data from a total of 73 females diagnosed as healthy (n = 42), or with functional dysphonia (n = 15) or with unilateral vocal fold paralysis (n = 16), were quantitatively analyzed. Glottal area waveforms (GAW) and left and right hemi-GAWs (hGAW) were extracted from the HSV recordings. Based on Poincare sections through phase space-embedded signals, two novel quantitative parameters were computed: the phasegram entropy (PE) and the phasegram complexity estimate (PCE), inspired by signal entropy and correlation dimension computation, respectively. Results Both PE and PCE assumed higher average values (suggesting more irregular vibrations) for the pathological as compared with the healthy participants, thus significantly discriminating healthy group from the paralysis group ( P = 0.02 for both PE and PCE). Comparisons of individual PE or PCE data for the left and the right hGAW within each subject resulted in asymmetry measures for the regularity of vocal fold vibration. The PCE-based asymmetry measure revealed significant differences between the healthy group and the paralysis group ( P = 0.03). Conclusions Quantitative phasegram analysis of GAW and hGAW data is a promising tool for the automated processing of HSV data in research and in clinical practice.

An important step in establishing the diagnostic potential for emerging optical imaging techniques is accurate registration between imaging data and the corresponding tissue histopathology typically used as gold standard in clinical diagnostics. We present a method to precisely register data acquired with a point-scanning spectroscopic imaging technique from fresh surgical tissue specimen blocks with corresponding histological sections. Using a visible aiming beam to augment point-scanning multispectral time-resolved fluorescence spectroscopy on video images, we evaluate two different markers for the registration with histology: fiducial markers using a 405-nm CW laser and the tissue block's outer shape characteristics. We compare the registration performance with benchmark methods using either the fiducial markers or the outer shape characteristics alone to a hybrid method using both feature types. The hybrid method was found to perform best reaching an average error of 0.78±0.67 mm. This method provides a profound framework to validate diagnostical abilities of optical fiber-based techniques and furthermore enables the application of supervised machine learning techniques to automate tissue characterization.

Objective This work presents a computer-based approach to analyze the two-dimensional vocal fold dynamics of endoscopic high-speed videos, and constitutes an extension and generalization of a previously proposed wavelet-based procedure. While most approaches aim for analyzing sustained phonation conditions, the proposed method allows for a clinically adequate analysis of both dynamic as well as sustained phonation paradigms. Materials and methods The analysis procedure is based on a spatio-temporal visualization technique, the phonovibrogram, that facilitates the documentation of the visible laryngeal dynamics. From the phonovibrogram, a low-dimensional set of features is computed using a principle component analysis strategy that quantifies the type of vibration patterns, irregularity, lateral symmetry and synchronicity, as a function of time. Two different test bench data sets are used to validate the approach: ( I ) 150 healthy and pathologic subjects examined during sustained phonation. ( II ) 20 healthy and pathologic subjects that were examined twice: during sustained phonation and a glissando from a low to a higher fundamental frequency. In order to assess the discriminative power of the extracted features, a Support Vector Machine is trained to distinguish between physiologic and pathologic vibrations. The results for sustained phonation sequences are compared to the previous approach. Finally, the classification performance of the stationary analyzing procedure is compared to the transient analysis of the glissando maneuver. Results For the first test bench the proposed procedure outperformed the previous approach (proposed feature set: accuracy: 91.3%, sensitivity: 80%, specificity: 97%, previous approach: accuracy: 89.3%, sensitivity: 76%, specificity: 96%). Comparing the classification performance of the second test bench further corroborates that analyzing transient paradigms provides clear additional diagnostic value (glissando maneuver: accuracy: 90%, sensitivity: 100%, specificity: 80%, sustained phonation: accuracy: 75%, sensitivity: 80%, specificity: 70%). Conclusions The incorporation of parameters describing the temporal evolvement of vocal fold vibration clearly improves the automatic identification of pathologic vibration patterns. Furthermore, incorporating a dynamic phonation paradigm provides additional valuable information about the underlying laryngeal dynamics that cannot be derived from sustained conditions. The proposed generalized approach provides a better overall classification performance than the previous approach, and hence constitutes a new advantageous tool for an improved clinical diagnosis of voice disorders.

About two thirds of laryngeal cancers originate at the vocal cords. Early-stage detection of malignant vocal fold alterations, including a discrimination of premalignant lesions, represents a major challenge in laryngology as precancerous vocal fold lesions and small carcinomas are difficult to distinguish by means of regular endoscopy only. We report a procedure to discriminate between malignant and precancerous lesions by measuring the characteristics of vocal fold dynamics by means of a computerized analysis of laryngeal high-speed videos. Ten patients with squamous cell T1a carcinoma, ten with precancerous lesions with hyperkeratosis, and ten subjects without laryngeal disease underwent high-speed laryngoscopy yielding 4,000 images per second. By means of wavelet-based phonovibrographic analysis, a set of three clinically meaningful vibratory measures was extracted from the videos comprising a total number of 15,000 video frames. Statistical analysis (ANOVA with post hoc two-sided t tests, P < 0.05) revealed that vocal fold dynamics is significantly affected in the presence of precancerous lesions and T1a carcinoma. On the basis of the three measures, a discriminating pattern was extracted using a support vector machine-learning algorithm performing an individual classification in respect to the different clinical groups. By applying a leave-one-out cross-validation strategy, we could show that the proposed measures discriminate with a very high performance between precancerous lesions and T1a carcinoma (sensitivity, 100%; specificity, 100%). Although a large-scale study will be necessary to confirm clinical significance, the set of vibratory measures derived in this study may be applicable to improve the accuracy and reliability of noninvasive diagnostics of vocal fold lesions. Cancer Res; 75(1); 31–39. ©2014 AACR.

Ten years of data on the reproductive biology of the Chacoan peccary (Catagonus wagneri) were analyzed to determine average litter size, sex ratio, timing of births, and individual and age-specific fecundity. Data were obtained from a captive herd of chacoan peccaries located in the western Paraguayan Chaco. Births peak in the austral spring months of September, October, and November, with fewer litters born during the dry season months of June, July, and August. The average litter size was 2.4 with a sex ratio of 56:44 (M:F). There was no significant difference in litter size among individual females or among females of different ages. Finally, there was some individual variation in the age at which sows produced their first litter, but no discernible variation among sows in the average time between litters. Zoo Biol 16:301–307, 1997. © 1997 Wiley-Liss, Inc.

Direct observation of vocal fold vibration is indispensable for a clinical diagnosis of voice disorders. Among current imaging techniques, high-speed videoendoscopy constitutes a state-of-the-art method capturing several thousand frames per second of the vocal folds during phonation. Recently, a method for extracting descriptive features from phonovibrograms, a two-dimensional image containing the spatio-temporal pattern of vocal fold dynamics, was presented. The derived features are closely related to a clinically established protocol for functional assessment of pathologic voices. The discriminative power of these features for different pathologic findings and configurations has not been assessed yet. In the current study, a collective of 220 subjects is considered for two- and multi-class problems of healthy and pathologic findings. The performance of the proposed feature set is compared to conventional feature reduction routines and was found to clearly outperform these. As such, the proposed procedure shows great potential for diagnostical issues of vocal fold disorders.

Recently, endoscopic high-speed laryngoscopy has been established for commercial use as a state-of-the-art technique to examine vocal fold kinematics. Since modern cameras provide sampling rates of several thousand frames per second, a high volume of data has to be considered for visual and objective analysis. A method for visualizing endoscopic high speed videos in three-dimensional cycle-based graphs combining and extending the approaches of phonovibrograms and electroglottographic wavegrams is presented. To build a phonovibrographic wavegram, individual cycles of a phonovibrogram are segmented, normalized in cycle duration, and concatenated over time. For analyzing purposes, the emerging three-dimensional scalar field is visualized with different rendering techniques providing information of different aspects of vocal fold kinematics. The phonovibrographic wavegram incorporates information about the glottal closure type, size, and location of the amplitudes, symmetry, periodicity, and phase information. The potential of the approach to visualize the characteristics of vocal fold vibration in a compact and intuitive way is demonstrated within two healthy and three pathologic subjects. The phonovibrographic wavegram allows a comprehensive analysis of vocal fold kinematics and reveals information that remains hidden with other visualization techniques.

Recently, endoscopic high-speed laryngoscopy has been established for commercial use and constitutes a state-of-the-art technique to examine vocal fold dynamics. Despite overcoming many limitations of commonly applied stroboscopy it has not gained widespread clinical application, yet. A major drawback is a missing methodology of extracting valuable features to support visual assessment or computer-aided diagnosis. In this paper a compact and descriptive feature set is presented. The feature extraction routines are based on two-dimensional color graphs called phonovibrograms (PVG). These graphs contain the full spatio-temporal pattern of vocal fold dynamics and are therefore suited to derive features that comprehensively describe the vibration pattern of vocal folds. Within our approach, clinically relevant features such as glottal closure type, symmetry and periodicity are quantified in a set of 10 descriptive features. The suitability for classification tasks is shown using a clinical data set comprising 50 healthy and 50 paralytic subjects. A classification accuracy of 93.2% has been achieved.