Search

To evaluate the improvement of radiologist performance in detecting bone metastases at follow up low-dose computed tomography (CT) by using a temporal subtraction (TS) technique based on an advanced nonrigid image registration algorithm.Twelve patients with bone metastases (males, 5; females, 7; mean age, 64.8 ± 7.6 years; range 51-81 years) and 12 control patients without bone metastases (males, 5; females, 7; mean age, 64.8 ± 7.6 years; 51-81 years) were included, who underwent initial and follow-up CT examinations between December 2005 and July 2016. Initial CT images were registered to follow-up CT images by the algorithm, and TS images were created. Three radiologists independently assessed the bone metastases with and without the TS images. The reader averaged jackknife alternative free-response receiver operating characteristics figure of merit was used to compare the diagnostic accuracy.The reader-averaged values of the jackknife alternative free-response receiver operating characteristics figures of merit (θ) significantly improved from 0.687 for the readout without TS and 0.803 for the readout with TS (P value = .031. F statistic = 5.24). The changes in the absolute value of CT attenuations in true-positive lesions were significantly larger than those in false-negative lesions (P < .001). Using TS, segment-based sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of the readout with TS were 66.7%, 98.9%, 94.4%, 90.9%, and 94.8%, respectively.The TS images can significantly improve the radiologist's performance in the detection of bone metastases on low-dose and relatively thick-slice CT.

Intraoperative applications of near-infrared (NIR) fluorescent contrast agents can be aided by instrumentation capable of merging the view of surgical field with that of NIR fluorescence. We demonstrate augmented microscopy, an intraoperative imaging technique in which bright-field (real) and electronically processed NIR fluorescence (synthetic) images are merged within the optical path of a stereomicroscope. Under luminance of 100,000 lx, representing typical illumination of the surgical field, the augmented microscope detects 189 nM concentration of indocyanine green and produces a composite of the real and synthetic images within the eyepiece of the microscope at 20 fps. Augmentation described here can be implemented as an add-on module to visualize NIR contrast agents, laser beams, or various types of electronic data within the surgical microscopes commonly used in neurosurgical, cerebrovascular, otolaryngological, and ophthalmic procedures.

Intravascular ultrasound (IVUS) imaging is extremely important for detection and characterization of high-risk atherosclerotic plaques as well as gastrointestinal diseases. Recently, intravascular photoacoustic (IVPA) imaging has been used to differentiate the composition of biological tissues with high optical contrast and ultrasonic resolution. The combination of these imaging techniques could provide morphological information and molecular screening to characterize abnormal tissues, which would help physicians to ensure vital therapeutic value and prognostic significance for patients before commencing therapy. In this study, integration of a high-frequency IVUS imaging catheter (45MHz, single-element, unfocused, 0.7mm in diameter) with a multi-mode optical fiber (0.6mm in core diameter, 0.22 NA), an integrated intravascular ultrasonic-photoacoustic (IVUP) imaging catheter, was developed to provide spatial and functional information on light distribution in a turbid sample. Simultaneously, IVUS imaging was co-registered to IVPA imaging to construct 3D volumetric sample images. In a phantom study, a polyvinyl alcohol (PVA) tissue-mimicking arterial vessel phantom with indocyanine green (ICG) and methylene blue (MB) inclusion was used to demonstrate the feasibility of mapping the biological dyes, which are used in cardiovascular and cancer diagnostics. For the ex vivo study, an excised sample of pig intestine with ICG was utilized to target the biomarkers present in the gastrointestinal tumors or the atherosclerotic plaques with the proposed hybrid technique. The results indicated that IVUP endoscope with the 2.2-mm diameter catheter could be a useful tool for medical imaging., (Copyright © 2015 Elsevier Ltd. All rights reserved.)

In this paper, we review the recent trends and advancements on correlation-based pattern recognition and tracking in forward-looking infrared (FLIR) imagery. In particular, we discuss matched filter-based correlation techniques for target detection and tracking which are widely used for various real time applications. We analyze and present test results involving recently reported matched filters such as the maximum average correlation height (MACH) filter and its variants, and distance classifier correlation filter (DCCF) and its variants. Test results are presented for both single/multiple target detection and tracking using various real-life FLIR image sequences.

We present a way to generate acousto-optical signals in timovssue-like media with nanosecond laser pulses. Our method is based on recording and analyzing speckle patterns formed by interaction of nanosecond laser pulses with tissue, without and with simultaneous application of ultrasound. Stroboscopic application allows visualizing the temporal behavior of speckles while the ultrasound is propagating through the medium. We investigate two ways of quantifying the acousto-optic effect, viz. adding and subtracting speckle patterns obtained at various ultrasound phases. Both methods are compared with the existing speckle contrast method using a 2D scan and are found to perform similarly. Our method gives outlook on overcoming the speckle decorrelation problem in acousto-optics, and therefore brings in-vivo acousto-optic measurements one step closer. Furthermore it enables combining acousto-optics and photoacoustics in one setup with a single laser.

One of the most promising concepts of starlight suppression for direct imaging of exoplanets is flying a specially-shaped external occulter in formation with a space telescope. Here we present contrast performance verification of an occulter design scaled to laboratory-size using Fresnel numbers corresponding to the space design. Experimental design innovations include usage of an expanding beam to minimize phase aberrations, and an outer ring to minimize hard-edge diffraction effects. The apodizing performance of the optimized occulter edge is compared with a baseline case of a circular occulter and shown to result in contrast improvements. Experimental results in red monochromatic light show that the achieved laboratory contrast exceeds ten orders of magnitude, but with differences from the theoretical diffraction analysis limited by specular reflection from the mask edges.

We developed an integrated dual-modal photoacoustic and optical coherence tomography (PA-OCT) system using a single near-infrared supercontinuum laser source to simultaneously provide both optical absorption and scattering contrasts. A pulsed broadband supercontinuum source was generated by a pulsed Nd:YAG laser and a photonic-crystal fiber. When we imaged two colored hairs, the black hair was visible in both PA and OCT images, whereas the white hair was only mapped in the OCT image. The single laser source will potentially allow us to implement relatively simple, cheap, and compact dual-modal PA-OCT systems, which are key criteria for fast clinical translation and commercialization.

Photoacoustic microscopy (PAM) is sensitive to optical absorption, while optical coherence tomography (OCT) is based on optical backscattering. Combining PAM and OCT can provide complementary information about biological tissue. Here we present a combined optical-resolution PAM (ORPAM) and OCT system that is integrated through a miniature probe with an overall diameter of 2.3 mm, suitable for insertion through a standard endoscopic or laparoscopic port during minimally invasive surgery or endoscopic exam. The hybrid probe consists of a common optical path for OCT (light delivery/detection) and ORPAM (light excitation) and a 10 MHz unfocused ultrasound transducer for photoacoustic detection. The combined system yields a lateral resolution of 15 μm for both ORPAM and OCT.

Purpose: To evaluate the accuracy of a novel combined electromagnetic (EM) navigation/image fusion system for biopsy of small lesions., Materials and Methods: Using ultrasound (US) guidance, metallic (2 × 1 mm) targets were imbedded in the paraspinal muscle (n = 28), kidney (n = 18), and liver (n = 4) of four 55- to 65-kg pigs. Baseline helical computed tomography (CT) imaging (Brilliance; Philips) identified these biopsy targets and six and nine cutaneous fiducial markers. CT data were imported into a MyLab Twice system (Esaote, Genoa, Italy) for CT/US image fusion. After verification of successful image fusion, baseline registration error and respiratory motion error were assessed by documenting deviation of the US and CT position of the targets in real time. Biopsy targeting was subsequently performed under conditions of normal respiratory using 15-cm 16G eTrax needles (Civco). To mimic the conditions of poor US visualization, only reconstructed CT information was displayed during biopsy. Accuracy of targeting was measured by repeat CT scanning as the distance of the needle tip to the target center. Targeting accuracy of free-hand vs. guided technique, and electromagnetic (EM) sensor positioning (ie, on the hub or within the needle stylus tip) were evaluated., Results: In muscle, needle registration error was 0.9 ± 1.2 mm and respiratory motion error 4.0 ± 1.0 mm. Target accuracy was 4.0 ± 3.2 mm when an EM sensor was imbedded in the needle tip. Yet, with the EM sensor back on the needle hub, greater targeting accuracy was achieved using an US guide (3.2 ± 1.6 mm) vs. freehand (5.7 ± 3.2 mm, P = .04). For kidney, registration error was 1.8 ± 1.7 mm and respiratory motion error 4.9 ± 1.0 mm. For the deeper kidney targets, target accuracy was 4.4 ± 3.2 mm with a tip EM sensor, which was an improvement over the hub EM sensor positioning (9.3 ± 4.6 mm; P < .01). An additional source of fusion error was noted for liver. Beyond 17 ± 1 mm of respiratory motion, targets were observed to move >3 cm with US transducer/needle compression resulting in 14 ± 1.4 mm targeting accuracy., Conclusions: A combined image-fusion/EM tracking platform can provide a high degree of needle placement accuracy (<5 mm) when targeting small lesions. Results fall within accuracy of respiratory error; with best results obtained by incorporating an EM sensor into the tip of the biopsy system., (Copyright © 2013. Published by Elsevier Inc.)

We have implemented a deterministic method for solving the phase problem in hard x-ray in-line holography which overcomes the twin image problem. The phase distribution in the detector plane is retrieved by using two images with slightly different Fresnel numbers. We then use measured intensities and reconstructed phases in the detection plane to compute the exit wave in the sample plane. No further a priori information like a limited support or the assumption of pure phase objects is necessary so that it can be used for a wide range of complex samples. Using a nano-focused hard x-ray beam half period resolutions better than 30 nm are achieved.

The multimodal optical imaging technique, which utilizes nonlinear and linear optical processes, plays an important role in biological and biomedical research. As second-order nonlinear phenomenon, the two-photon luminescence (TPL) results from the nonlinear excitation of fluorescent molecules, while the second harmonic generation (SHG) depends on the second order nonlinear polarization, orientation, and noncentrosymmetric properties of molecules. In contrast, the linear resonance light scattering (RLS) occurs when the molecules are excited by a light beam with a wavelength close to their absorption bands. Since SHG, TPL, and RLS involve different kinds of optical processes, they might be used in parallel to provide complementary information about the structure and function of cells and tissues. Herein, we develop for the first time a multimodal optical microscopy with the capability of simultaneous SHG, TPL, and RLS imaging. We analyze theoretically and demonstrate experimentally the near-infrared irradiation-induced SHG, TPL, and RLS from the gold nanorods with nanometer spatial resolution. With the gold nanorods as the contrast agents, we further demonstrate the simultaneous SHG, TPL, and RLS imaging of A431 human epithelial skin cancer cells. This multimodal optical microscopy might provide a reliable and complementary approach for biological and biomedical research.

We introduce an integration of dynamic light scattering (DLS) and optical coherence tomography (OCT) for high-resolution 3D imaging of heterogeneous diffusion and flow. DLS analyzes fluctuations in light scattered by particles to measure diffusion or flow of the particles, and OCT uses coherence gating to collect light only scattered from a small volume for high-resolution structural imaging. Therefore, the integration of DLS and OCT enables high-resolution 3D imaging of diffusion and flow. We derived a theory under the assumption that static and moving particles are mixed within the OCT resolution volume and the moving particles can exhibit either diffusive or translational motion. Based on this theory, we developed a fitting algorithm to estimate dynamic parameters including the axial and transverse velocities and the diffusion coefficient. We validated DLS-OCT measurements of diffusion and flow through numerical simulations and phantom experiments. As an example application, we performed DLS-OCT imaging of the living animal brain, resulting in 3D maps of the absolute and axial velocities, the diffusion coefficient, and the coefficient of determination.

A major challenge in high-resolution intravital confocal and multiphoton microscopy is physiologic tissue movement during image acquisition. Of the various physiological sources of movement, respiration has arguably the largest and most wide-ranging effect. We describe a technique for achieving stabilized microscopy imaging using a dual strategy. First, we designed a mechanical stabilizer for constraining physical motion; this served to simultaneously increase the in-focus range over which data can be acquired as well as increase the reproducibility of imaging a certain position within each confocal imaging plane. Second, by implementing a retrospective breathing-gated imaging modality, we performed selective image extraction gated to a particular phase of the respiratory cycle. Thanks to the high reproducibility in position, all gated images presented a high degree of correlation over time. The images obtained using this technique not only showed significant improvements over images acquired without the stabilizer, but also demonstrated accurate in vivo imaging during longitudinal studies. The described methodology is easy to implement with any commercial imaging system, as are used by most biological imaging laboratories, and can be used for both confocal and multiphoton laser scanning microscopy.

A Near Infrared Spectral Tomography (NIRST) system has been developed and integrated into a commercial Digital Breast Tomosynthesis (DBT) scanner to allow structural and functional imaging of breast in vivo. The NIRST instrument uses an 8-wavelength continuous wave (CW) laser-based scanning source assembly and a 75-element silicon photodiode solid-state detector panel to produce dense spectral and spatial projection data from which spectrally constrained 3D tomographic images of tissue chromophores are produced. Integration of the optical imaging system into the DBT scanner allows direct co-registration of the optical and DBT images, while also facilitating the synergistic use of x-ray contrast as anatomical priors in optical image reconstruction. Currently, the total scan time for a combined NIRST-DBT exam is ~50s with data collection from 8 wavelengths in the optical scan requiring ~42s to complete. The system was tested in breast simulating phantoms constructed using intralipid and blood in an agarose matrix with a 3 cm x 2 cm cylindrical inclusion at 1 cm depth from the surface. Diffuse image reconstruction of total hemoglobin (HbT) concentration resulted in accurate recovery of the lateral size and position of the inclusion to within 6% and 8%, respectively. Use of DBT structural priors in the NIRST reconstruction process improved the quantitative accuracy of the HbT recovery, and led to linear changes in imaged versus actual contrast, underscoring the advantages of dual-modality optical imaging approaches. The quantitative accuracy of the system can be further improved with independent measurements of scattering properties through integration of frequency or time domain data.

We have accomplished simultaneous spectral domain optical coherence tomography (SD-OCT) and autofluorescence (AF) microscopy with a broadband light source centered at 415 nm. The light source was provided by frequency-doubling of an ultra-fast broadband Ti:Sapphire laser. With a bandwidth of 8 nm, the visible SD-OCT achieved a depth resolution of ~12  μm. Since the two imaging modalities are provided by the same group of photons, their images are intrinsically registered. The dual-modal system is capable of providing OCT imaging and molecular contrasts simultaneously. The imaging system was tested on imaging biological samples ex vivo and in vivo.

In optical-resolution photo-acoustic microscopy (OR-PAM), data acquisition time is limited by both laser pulse repetition rate (PRR) and scanning speed. Optical-scanning offers high speed, but limited, field of view determined by ultrasound transducer sensitivity. In this paper, we propose a hybrid optical and mechanical-scanning OR-PAM system with mosaic data acquisition and processing. The system employs fast-scanning mirrors and a diode-pumped, nanosecond-pulsed, Ytterbium-doped, 532-nm fiber laser with PRR up to 600 kHz. Data from a sequence of image mosaic patches is acquired systematically, at predetermined mechanical scanning locations, with optical scanning. After all imaging locations are covered, a large panoramic scene is generated by stitching the mosaic patches together. Our proposed system is proven to be at least 20 times faster than previous reported OR-PAM systems.

A side-viewing, 2.3-mm diameter, surface magnifying chromoendoscopy-optical coherence tomography (SMC-OCT) endoscope has been designed for simultaneous, nondestructive surface fluorescence visualization and cross-sectional imaging. We apply this endoscope to in vivo examination of the mouse colon. A 30,000 element fiber bundle is combined with single mode fibers, for SMC and OCT imaging, respectively. The distal optics consist of a gradient-index lens and spacer to provide a 1× magnification at a working distance of 1.58 mm in air, necessary to image the sample through a 0.23-mm thick outer glass envelope, and an aluminized right-angle prism fixed to the distal end of the gradient-index lens assembly. The resulting 1∶1 imaging system is capable of 3.9-µm lateral and 2.3-µm axial resolution in the OCT channel, and 125-lp/mm resolution across a 0.70-mm field of view in the SMC channel. The endoscope can perform high contrast crypt visualization, molecular imaging, and cross-sectional imaging of colon microstructure.

Although several strategies exist for a minimal-invasive treatment of patients with lung failure, the mortality rate of acute respiratory distress syndrome still reaches 30% at minimum. This striking number indicates the necessity of understanding lung dynamics on an alveolar level. To investigate the dynamical behavior on a microscale, we used three-dimensional geometrical and functional imaging to observe tissue parameters including alveolar size and length of embedded elastic fibers during ventilation. We established a combined optical coherence tomography (OCT) and confocal fluorescence microscopy system that is able to monitor the distension of alveolar tissue and elastin fibers simultaneously within three dimensions. The OCT system can laterally resolve a 4.9 μm line pair feature and has an approximately 11 μm full-width-half-maximum axial resolution in air. confocal fluorescence microscopy visualizes molecular properties of the tissue with a resolution of 0.75 μm (laterally), and 5.9 μm (axially) via fluorescence detection of the dye sulforhodamine B specifically binding to elastin. For system evaluation, we used a mouse model in situ to perform lung distension by application of different constant pressure values within the physiological regime. Our method enables the investigation of alveolar dynamics by helping to reveal basic processes emerging during artificial ventilation and breathing.

In this paper, we report, to the best of our knowledge, a unique field-programmable gate array (FPGA)-based reconfigurable processor for real-time interlaced co-registered ultrasound and photoacoustic imaging and its application in imaging tumor dynamic response. The FPGA is used to control, acquire, store, delay-and-sum, and transfer the data for real-time co-registered imaging. The FPGA controls the ultrasound transmission and ultrasound and photoacoustic data acquisition process of a customized 16-channel module that contains all of the necessary analog and digital circuits. The 16-channel module is one of multiple modules plugged into a motherboard; their beamformed outputs are made available for a digital signal processor (DSP) to access using an external memory interface (EMIF). The FPGA performs a key role through ultrafast reconfiguration and adaptation of its structure to allow real-time switching between the two imaging modes, including transmission control, laser synchronization, internal memory structure, beamforming, and EMIF structure and memory size. It performs another role by parallel accessing of internal memories and multi-thread processing to reduce the transfer of data and the processing load on the DSP. Furthermore, because the laser will be pulsing even during ultrasound pulse-echo acquisition, the FPGA ensures that the laser pulses are far enough from the pulse-echo acquisitions by appropriate time-division multiplexing (TDM). A co-registered ultrasound and photoacoustic imaging system consisting of four FPGA modules (64-channels) is constructed, and its performance is demonstrated using phantom targets and in vivo mouse tumor models.

We developed a multimodality fluorescence and optical coherence tomography probe based on a double-clad fiber (DCF) combiner. The probe is composed of a DCF combiner, grin lens, and micromotor in the distal end. An integrated swept-source optical coherence tomography and fluorescence intensity imaging system was developed based on the combined probe for the early diagnoses of atherosclerosis. This system is capable of real-time data acquisition and processing as well as image display. For fluorescence imaging, the inflammation of atherosclerosis and necrotic core formed with the annexin V-conjugated Cy5.5 were imaged. Ex vivo imaging of New Zealand white rabbit arteries demonstrated the capability of the combined system.

We developed a novel trimodality system for human breast imaging by integrating photoacoustic (PA) and thermoacoustic (TA) imaging techniques into a modified commercial ultrasound scanner. Because light was delivered with an optical assembly placed within the microwave antenna, no mechanical switching between the microwave and laser sources was needed. Laser and microwave excitation pulses were interleaved to enable PA and TA data acquisition in parallel at a rate of 10 frames per second. A tube (7 mm inner diameter) filled with oxygenated bovine blood or 30 mM methylene blue dye was successfully detected in PA images in chicken breast tissue at depths of 6.6 and 8.4 cm, respectively, for the first time. The SNRs at these depths reached ∼24 and ∼15 dB, respectively, by averaging 200 signal acquisitions. Similarly, a tube (13 mm inner diameter) filled with saline solution (0.9%) at a depth of 4.4 cm in porcine fat tissue was successfully detected in TA images. The PA axial, lateral, and elevational resolutions were 640 μm, 720 μm, and 3.5 mm, respectively, suitable for breast cancer imaging. A PA noise-equivalent sensitivity to methylene blue solution of 260 nM was achieved in chicken tissue at a depth of 3.4 cm.

Background subtraction is considered the first processing stage in video surveillance systems, and consists of determining objects in movement in a scene captured by a static camera. It is an intensive task with a high computational cost. This work proposes an embedded novel architecture on FPGA which is able to extract the background on resource-limited environments and offers low degradation (produced because of the hardware-friendly model modification). In addition, the original model is extended in order to detect shadows and improve the quality of the segmentation of the moving objects. We have analyzed the resource consumption and performance in Spartan3 Xilinx FPGAs and compared to others works available on the literature, showing that the current architecture is a good trade-off in terms of accuracy, performance and resources utilization. With less than a 65% of the resources utilization of a XC3SD3400 Spartan-3A low-cost family FPGA, the system achieves a frequency of 66.5 MHz reaching 32.8 fps with resolution 1,024 × 1,024 pixels, and an estimated power consumption of 5.76 W.

The combination of two-photon microscopy (TPM) and optical coherence tomography (OCT) is useful in conducting in-vivo tissue studies, because they provide complementary information regarding tissues. In the present study, we developed a new combined system using separate light sources and scanners for individually optimal imaging conditions. TPM used a Ti-Sapphire laser and provided molecular and cellular information in microscopic tissue regions. Meanwhile, OCT used a wavelength-swept source centered at 1300 nm and provided structural information in larger tissue regions than TPM. The system was designed to do simultaneous imaging by combining light from both sources. TPM and OCT had the field of view values of 300 μm and 800 μm on one side respectively with a 20x objective. TPM had resolutions of 0.47 μm and 2.5 μm in the lateral and axial directions respectively, and an imaging speed of 40 frames/s. OCT had resolutions of 5 μm and 8 μm in lateral and axial directions respectively, a sensitivity of 97dB, and an imaging speed of 0.8 volumes per second. This combined system was tested with simple microsphere specimens, and was then applied to image small intestine and ear tissues of mouse models ex-vivo. Molecular, cellular, and structural information of the tissues were visualized using the proposed combined system.

The high sensitivity of fluorescence imaging enables the detection of molecular processes in living organisms. However, diffuse light propagation in tissue prevents accurate recovery of tomographic information on fluorophore distribution for structures embedded deeper than 0.5 mm. Combining optical with magnetic resonance imaging (MRI) provides an accurate anatomical reference for fluorescence imaging data and thereby enables the correlation of molecular with high quality structural/functional information. We describe an integrated system for small animal imaging incorporating a noncontact fluorescence molecular tomography (FMT) system into an MRI detector. By adopting a free laser beam design geometrical constraints imposed by the use of optical fibers could be avoided allowing for flexible fluorescence excitation schemes. Photon detection based on a single-photon avalanche diode array enabled simultaneous FMT/MRI measurements without interference between modalities. In vitro characterization revealed good spatial accuracy of FMT data and accurate quantification of dye concentrations. Feasibility of FMT/MRI was demonstrated in vivo by simultaneous assessment of protease activity and tumor morphology in murine colon cancer xenografts.

This paper addresses the issue of matching rigid and articulated shapes through probabilistic point registration. The problem is recast into a missing data framework where unknown correspondences are handled via mixture models. Adopting a maximum likelihood principle, we introduce an innovative EM-like algorithm, namely, the Expectation Conditional Maximization for Point Registration (ECMPR) algorithm. The algorithm allows the use of general covariance matrices for the mixture model components and improves over the isotropic covariance case. We analyze in detail the associated consequences in terms of estimation of the registration parameters, and propose an optimal method for estimating the rotational and translational parameters based on semidefinite positive relaxation. We extend rigid registration to articulated registration. Robustness is ensured by detecting and rejecting outliers through the addition of a uniform component to the Gaussian mixture model at hand. We provide an in-depth analysis of our method and compare it both theoretically and experimentally with other robust methods for point registration.

The ray projection and its application to recovering a projective geometric transformation and an affine lighting change between two objects are mathematically studied. A novel technique, viz., variable contour, is proposed for accurately evaluating the ray projection. Moreover, a novel flexible framework of the ray projection is devised for the joint recovery of the eight parameters for the projective transformation and the two parameters for the lighting change. Finally, the framework is experimentally evaluated by recovering a variety of geometric and photometric transformations between real images of indoor and outdoor scenes. Its robustness to image blur and occlusion is demonstrated. Its versatility is illustrated through matching different objects in different classes.

Object detection is challenging when the object class exhibits large within-class variations. In this work, we show that foreground-background classification (detection) and within-class classification of the foreground class (pose estimation) can be jointly learned in a multiplicative form of two kernel functions. Model training is accomplished via standard SVM learning. When the foreground object masks are provided in training, the detectors can also produce object segmentations. A tracking-by-detection framework to recover foreground state in video sequences is also proposed with our model. The advantages of our method are demonstrated on tasks of object detection, view angle estimation, and tracking. Our approach compares favorably to existing methods on hand and vehicle detection tasks. Quantitative tracking results are given on sequences of moving vehicles and human faces.

Extracting high-quality dynamic foreground layers from a video sequence is a challenging problem due to the coupling of color, motion, and occlusion. Many approaches assume that the background scene is static or undergoes the planar perspective transformation. In this paper, we relax these restrictions and present a comprehensive system for accurately computing object motion, layer, and depth information. A novel algorithm that combines different clues to extract the foreground layer is proposed, where a voting-like scheme robust to outliers is employed in optimization. The system is capable of handling difficult examples in which the background is nonplanar and the camera freely moves during video capturing. Our work finds several applications, such as high-quality view interpolation and video editing.

Modeling illumination effects and pose variations of a face is of fundamental importance in the field of facial image analysis. Most of the conventional techniques that simultaneously address both of these problems work with the Lambertian assumption and thus fall short of accurately capturing the complex intensity variation that the facial images exhibit or recovering their 3D shape in the presence of specularities and cast shadows. In this paper, we present a novel Tensor-Spline-based framework for facial image analysis. We show that, using this framework, the facial apparent BRDF field can be accurately estimated while seamlessly accounting for cast shadows and specularities. Further, using local neighborhood information, the same framework can be exploited to recover the 3D shape of the face (to handle pose variation). We quantitatively validate the accuracy of the Tensor Spline model using a more general model based on the mixture of single-lobed spherical functions. We demonstrate the effectiveness of our technique by presenting extensive experimental results for face relighting, 3D shape recovery, and face recognition using the Extended Yale B and CMU PIE benchmark data sets.

Recently proposed fast template matching techniques employ rejection schemes derived from lower bounds on the match measure. This paper generalizes that idea and shows that in addition to lower bounds, upper bounds on the match measure can be used to accelerate the search. An algorithm is proposed that utilizes both lower and upper bounds to detect the k best matches in an image. The performance of this dual-bound algorithm is guaranteed; it always detects the k best matches. Theoretical analysis and experimental results show that its runtime compares favorably with previously proposed real-time exact template-matching schemes.

We report multifunctional optical imaging using dye-coated gold nanorods. Three types of useful information, namely, Raman, fluorescence signals, and absorption contrast, can be obtained from a phantom experiment. These three kinds of information are detected in a nanoparticle-doped-phantom using diffuse optical imaging. Our novel nanoparticle could be used as a multimodality marker for future bioimaging applications.

We demonstrate quantitative functional imaging using image-guided near-infrared spectroscopy (IG-NIRS) implemented with the boundary element method (BEM) for reconstructing 3-D optical property estimates in breast tissue in vivo. A multimodality MRI-NIR system was used to collect measurements of light reflectance from breast tissue. The BEM was used to model light propagation in 3-D based only on surface discretization in order to reconstruct quantitative values of total hemoglobin (HbT), oxygen saturation, water, and scatter. The technique was validated in experimental measurements from heterogeneous breast-shaped phantoms with known values and applied to a total of seven subjects comprising six healthy individuals and one participant with cancer imaged at two time points during neoadjuvant chemotherapy. Using experimental measurements from a heterogeneous breast phantom, BEM for IG-NIRS produced accurate values for HbT in the inclusion with a <3% error. Healthy breast tissues showed higher HbT and water in fibroglandular tissue than in adipose tissue. In a subject with cancer, the tumor showed higher HbT compared to the background. HbT in the tumor was reduced by 9 μM during treatment. We conclude that 3-D MRI-NIRS with BEM provides quantitative and functional characterization of breast tissue in vivo through measurement of hemoglobin content. The method provides potentially complementary information to DCE-MRI for tumor characterization.

Recent advances in registration have extended intrasurgical image guidance from its origins in bone-based procedures to new applications in soft tissues, thus enabling visualization of spatial relationships between surgical instruments and subsurface structures before incisions begin. Preoperative images are generally registered to soft tissues through aligning segmented volumetric image data with an intraoperatively sensed cloud of organ surface points. However, there is currently no viable noncontact minimally invasive scanning technology that can collect these points through a single laparoscopic port, which limits wider adoption of soft-tissue image guidance. In this paper, we describe a system based on conoscopic holography that is capable of minimally invasive surface scanning. We present the results of several validation experiments scanning ex vivo biological and phantom tissues with a system consisting of a tracked, off-the-shelf, relatively inexpensive conoscopic holography unit. These experiments indicate that conoscopic holography is suitable for use with biological tissues, and can provide surface scans of comparable quality to existing clinically used laser range scanning systems that require open surgery. We demonstrate experimentally that conoscopic holography can be used to guide a surgical needle to desired subsurface targets with an average tip error of less than 3 mm.

A 3-D augmented reality navigation system using autostereoscopic images was developed for MRI-guided surgery. The 3-D images are created by employing an animated autostereoscopic image, integral videography (IV), which provides geometrically accurate 3-D spatial images and reproduces motion parallax without using any supplementary eyeglasses or tracking devices. The spatially projected 3-D images are superimposed onto the surgical area and viewed via a half-silvered mirror. A fast and accurate spatial image registration method was developed for intraoperative i.v. image-guided therapy. Preliminary experiments showed that the total system error in patient-to-image registration was 0.90 +/- 0.21 mm, and the procedure time for guiding a needle toward a target was shortened by 75%. An animal experiment was also conducted to evaluate the performance of the system. The feasibility studies showed that augmented reality of the image overlay system could increase the surgical instrument placement accuracy and reduce the procedure time as a result of intuitive 3-D viewing.

This paper demonstrates the first combination for wide-field surface plasmon (SP) phase microscopy and SP-enhanced fluorescence microscopy to image living cells' contacts on the surface of a bio-substrate simultaneously. The phase microscopy with a phase-shift interferometry and common-path optical setup can provide high-sensitivity phase information in long-term stability. Simultaneously, the fluorescence microscopy with the enhancement of a local electromagnetic field can supply bright fluorescent images. The combined microscope imposes a high numerical aperture objective upon the excitation of surface plasmon through a silver film with a thickness of 30 nm. The developed SP microscope is successfully applied to the real-time bright observation of the transfected fluorescence of living cells localized near the cell membrane on the bio-substrate and the high-sensitivity phase image of the cell-substrate contacts at the same time.

This paper describes a precision-guided surgical navigation system for minimally invasive surgery. The system combines a laser guidance technique with a three-dimensional (3D) autostereoscopic image overlay technique. Images of surgical anatomic structures superimposed onto the patient are created by employing an animated imaging method called integral videography (IV), which can display geometrically accurate 3D autostereoscopic images and reproduce motion parallax without the need for special viewing or tracking devices. To improve the placement accuracy of surgical instruments, we integrated an image overlay system with a laser guidance system for alignment of the surgical instrument and better visualization of patient's internal structure. We fabricated a laser guidance device and mounted it on an IV image overlay device. Experimental evaluations showed that the system could guide a linear surgical instrument toward a target with an average error of 2.48 mm and standard deviation of 1.76 mm. Further improvement to the design of the laser guidance device and the patient-image registration procedure of the IV image overlay will make this system practical; its use would increase surgical accuracy and reduce invasiveness., (Copyright 2009 Elsevier Ltd. All rights reserved.)

A sub-pixel image shifter is presented, for use in enhancing the spatial resolution of digital image sensors by combining multiple displaced sub-images using a super-resolution (SR) algorithm. The device uses the walk-off phenomenon in birefringent crystals to separate images with opposite polarizations by a sub-pixel displacement. A liquid crystal (LC) waveplate plus a polarizer can then select the specific image to be exposed, with fast, non-mechanical control. This cascaded device, comprising two sapphire crystals, two LCs, and a single polarizer, is capable of 2-dimensional image shift with displacements of 0.5 pixels. The experimental results show that the image registration stability can be precisely controlled within 0.05 pixels and the contrast transfer function ratio of the SR image is enhanced by up to 1.36 times compared to the original captured image. Moreover, based on the fast transition time of LCs, the displaced sub-images can be recorded in video form with a frame rate of 40 fps.

Improved differential confocal microscopy is proposed to improve axial resolution and to enhance disturbance resistibility of confocal microscopy. The subtraction and sum values of the two defocusing detected signals are divided as the response function. Both ultrahigh signal-to-noise ratio (SNR) and wide range can be selectively obtained by controlling the defocusing amount of the two differential detectors more tightly with the reflectance disturbance resistibility. Since the detecting sensitivity of the proposed confocal microscopy is unrelated to the energy loss of the reflected beam, the multiplicative mode disturbance can be used to measure microstructures made of hybrid materials and overcome the power drift of a laser source during long scanning. In the case of ultrahigh SNR, the axial resolution reaches 1 nm when NA=0.75 and lambda=632.8 nm.

Unlabelled: We have constructed a dedicated breast PET/CT scanner capable of high-resolution functional and anatomic imaging. Here, we present an initial characterization of scanner performance during patient imaging., Methods: The system consisted of a lutetium oxyorthosilicate-based dual-planar head PET camera (crystal size, 3 x 3 x 20 mm) and 768-slice cone-beam CT. The position of the PET heads (separation and height) could be adjusted for varying breast dimensions. For scanning, the patient lay prone on a specialized bed and inserted a single pendent breast through an aperture in the table top. Compression of the breast as used in mammography is not required. PET and CT systems rotate in the coronal plane underneath the patient sequentially to collect fully tomographic datasets. PET images were reconstructed with the fully 3-dimensional maximum a posteriori method, and CT images were reconstructed with the Feldkamp algorithm, then spatially registered and fused for display. Phantom scans were obtained to assess the registration accuracy between PET and CT images and the influence of PET electronics and activity on CT image quality. We imaged 4 women with mammographic findings highly suggestive of breast cancer (breast imaging reporting and data system, category 5) in an ongoing clinical trial. Patients were injected with (18)F-FDG and imaged for 12.5 min per breast. From patient data, noise-equivalent counting rates and the singles-to-trues ratio (a surrogate for the randoms fraction) were calculated., Results: The average registration error between PET and CT images was 0.18 mm. PET electronics and activity did not significantly affect CT image quality. For the patient trial, biopsy-confirmed cancers were visualized on dedicated breast PET/CT on all patient scans, including the detection of ductal carcinoma in situ in 1 case. The singles-to-trues ratio was found to be inversely correlated with breast volume in the field of view, suggesting that larger breasts trend toward increased noise-equivalent counting rates for all other things equal., Conclusion: Scanning of the uncompressed breast with dedicated breast PET/CT can accurately visualize suspected lesions in 3 dimensions.

Both reflected confocal and multiphoton microscopy can have clinical diagnostic applications. The successful combination of both modalities in tissue imaging enables unique image contrast to be achieved, especially if a single laser excitation wavelength is used. We apply this approach for skin and corneal imaging using the 780-nm output of a femtosecond, titanium-sapphire laser. We find that the near-IR, reflected confocal (RC) signal is useful in characterizing refractive index varying boundaries in bovine cornea and porcine skin, while the multiphoton autofluorescence (MAF) and second-harmonic generation (SHG) intensities can be used to image cytoplasm and connective tissues (collagen), respectively. In addition, quantitative analysis shows that we are able to detect MAF from greater imaging depths than with the near-IR RC signal. Furthermore, by performing RC imaging at 488, 543, and 633 nm, we find that a longer wavelength leads to better image contrast for deeper imaging of the bovine cornea and porcine skin tissue. Finally, by varying power of the 780-nm source, we find that comparable RC image quality was achieved in the 2.7 to 10.7-mW range.

Rationale and Objectives: Magnetic resonance imaging with an endorectal coil allows high-resolution imaging of prostate cancer and the surrounding normal organs. These anatomic details can be used to direct radiotherapy. However, organ deformation introduced by the endorectal coil makes it difficult to register magnetic resonance images for treatment planning. In this study, plug-ins for the volume visualization software VolView were implemented on the basis of algorithms from the National Library of Medicine's Insight Segmentation and Registration Toolkit (ITK)., Materials and Methods: Magnetic resonance images of a phantom simulating human pelvic structures were obtained with and without the endorectal coil balloon inflated. The prostate not deformed by the endorectal balloon was registered to the deformed prostate using an ITK thin plate spline (TPS). This plug-in allows the use of crop planes to limit the deformable registration in the region of interest around the prostate. These crop planes restricted the support of the TPS to the area around the prostate, where most of the deformation occurred. The region outside the crop planes was anchored by grid points., Results: The TPS was more accurate in registering the local deformation of the prostate compared with a TPS variant, the elastic body spline. The TPS was also applied to register an in vivo T(2)-weighted endorectal magnetic resonance image. The intraprostatic tumor was accurately registered. This could potentially guide the boosting of intraprostatic targets. The source and target landmarks were placed graphically. This TPS plug-in allows the registration to be undone. The landmarks could be added, removed, and adjusted in real time and in three dimensions between repeated registrations., Conclusion: This interactive TPS plug-in allows a user to obtain a high level of accuracy satisfactory to a specific application efficiently. Because it is open-source software, the imaging community will be able to validate and improve the algorithm.

A new method for the detection of light traversing a diffuser/nondiffuser interface and its simultaneous determination for optical tomography is proposed, and the preliminary results are shown. The method is based on the use of a point detector and two uncoupled scanning systems-one for illumination and the other for registration-together with active modification of the optics guided by the surface topography to generate virtual detectors on the interface.

The procedures we propose make possible the mapping of two-dimensional (2-D) bioluminescence image (BLI) data onto a skin surface derived from a three-dimensional (3-D) anatomical modality [magnetic resonance (MR) or computed tomography (CT)] dataset. This mapping allows anatomical information to be incorporated into bioluminescence tomography (BLT) reconstruction procedures and, when applied using sources visible to both optical and anatomical modalities, can be used to evaluate the accuracy of those reconstructions. Our procedures, based on immobilization of the animal and a priori determined fixed projective transforms, should be more robust and accurate than previously described efforts, which rely on a poorly constrained retrospectively determined warping of the 3-D anatomical information. Experiments conducted to measure the accuracy of the proposed registration procedure found it to have a mean error of 0.36+/-0.23 mm. Additional experiments highlight some of the confounds that are often overlooked in the BLT reconstruction process, and for two of these confounds, simple corrections are proposed.

Lifetime imaging microscopy is a powerful tool to probe biological phenomena independent of luminescence intensity and fluorophore concentration. We describe time-resolved imaging of long-lifetime luminescence with an unmodified commercial laser scanning confocal/multiphoton microscope. The principle of the measurement is displacement of the detection pinhole to collect delayed luminescence from a position lagging the rasting laser beam. As proof of principle, luminescence from microspheres containing europium (Eu(3+)), a red emitting probe, was compared to that of short-lifetime green-fluorescing microspheres and/or fluorescein and rhodamine in solution. Using 720-nm two-photon excitation and a pinhole diameter of 1 Airy unit, the short-lifetime fluorescence of fluorescein, rhodamine and green microspheres disappeared much more rapidly than the long-lifetime phosphorescence of Eu(3+) microspheres as the pinhole was repositioned in the lagging direction. In contrast, repositioning of the pinhole in the leading and orthogonal directions caused equal loss of short- and long-lifetime luminescence. From measurements at different lag pinhole positions, a lifetime of 270 micros was estimated for the Eu(3+) microspheres, consistent with independent measurements. This simple adaptation is the basis for quantitative 3-D lifetime imaging microscopy.

The extensive development of image fusion techniques over the past 20 y has shown that the fusion of images from complementary modalities offers a more complete and accurate assessment of disease than do images from a single modality. Although software techniques have been successful in fusing images of the brain from different modalities, they have achieved rather limited success for other parts of the body. The recent introduction of technology that can acquire both anatomic and functional images in a single scan has addressed many of the limitations of software fusion. The combination of CT and PET was introduced commercially in 2001, followed by CT and SPECT in 2004. Clinical adoption of PET/CT has been surprisingly rapid, and despite continuing debate, the new technology has advanced the use of clinical molecular imaging, particularly for oncology.

This paper studies a family of distributions constructed from multivariate gamma distributions to model the statistical properties of multisensor synthetic aperture radar (SAR) images. These distributions referred to as multisensor multivariate gamma distributions (MuMGDs) are potentially interesting for detecting changes in SAR images acquired by different sensors having different numbers of looks. The first part of this paper compares different estimators for the parameters of MuMGDs. These estimators are based on the maximum likelihood principle, the method of inference function for margins, and the method of moments. The second part of the paper studies change detection algorithms based on the estimated correlation coefficient of MuMGDs. Simulation results conducted on synthetic and real data illustrate the performance of these change detectors.

Large core needle biopsy is a common procedure used to obtain histological samples when cancer is suspected in diagnostic breast images. The procedure is typically performed under image guidance, with freehand ultrasound and stereotactic mammography (SM) being the most common modalities used. To utilize the advantages of both modalities, a biopsy device combining three-dimensional ultrasound (3DUS) and digital SM imaging with computer-aided needle guidance was developed. An implementation of a stereo camera method was applied to SM calibration, providing a target localization error of 0.35 mm. The 3-D transformation between the two imaging modalities was then derived, with a target registration error of 0.52 mm. Finally, the needle guidance error of the device was evaluated using tissue-mimicking phantoms, showing a sample mean and standard deviation of 0.44 +/- 0.22 and 0.49 +/- 0.27 mm for targets planned from 3DUS and SM images, respectively. These results suggest that a biopsy procedure guided using this device would successfully sample breast lesions at a size greater than or equal to the smallest typically detected in mammographic screening (approximately 2 mm).