Search

In this paper, we report, to the best of our knowledge, a unique field-programmable gate array (FPGA)-based reconfigurable processor for real-time interlaced co-registered ultrasound and photoacoustic imaging and its application in imaging tumor dynamic response. The FPGA is used to control, acquire, store, delay-and-sum, and transfer the data for real-time co-registered imaging. The FPGA controls the ultrasound transmission and ultrasound and photoacoustic data acquisition process of a customized 16-channel module that contains all of the necessary analog and digital circuits. The 16-channel module is one of multiple modules plugged into a motherboard; their beamformed outputs are made available for a digital signal processor (DSP) to access using an external memory interface (EMIF). The FPGA performs a key role through ultrafast reconfiguration and adaptation of its structure to allow real-time switching between the two imaging modes, including transmission control, laser synchronization, internal memory structure, beamforming, and EMIF structure and memory size. It performs another role by parallel accessing of internal memories and multi-thread processing to reduce the transfer of data and the processing load on the DSP. Furthermore, because the laser will be pulsing even during ultrasound pulse-echo acquisition, the FPGA ensures that the laser pulses are far enough from the pulse-echo acquisitions by appropriate time-division multiplexing (TDM). A co-registered ultrasound and photoacoustic imaging system consisting of four FPGA modules (64-channels) is constructed, and its performance is demonstrated using phantom targets and in vivo mouse tumor models.

The high sensitivity of fluorescence imaging enables the detection of molecular processes in living organisms. However, diffuse light propagation in tissue prevents accurate recovery of tomographic information on fluorophore distribution for structures embedded deeper than 0.5 mm. Combining optical with magnetic resonance imaging (MRI) provides an accurate anatomical reference for fluorescence imaging data and thereby enables the correlation of molecular with high quality structural/functional information. We describe an integrated system for small animal imaging incorporating a noncontact fluorescence molecular tomography (FMT) system into an MRI detector. By adopting a free laser beam design geometrical constraints imposed by the use of optical fibers could be avoided allowing for flexible fluorescence excitation schemes. Photon detection based on a single-photon avalanche diode array enabled simultaneous FMT/MRI measurements without interference between modalities. In vitro characterization revealed good spatial accuracy of FMT data and accurate quantification of dye concentrations. Feasibility of FMT/MRI was demonstrated in vivo by simultaneous assessment of protease activity and tumor morphology in murine colon cancer xenografts.

Recent advances in registration have extended intrasurgical image guidance from its origins in bone-based procedures to new applications in soft tissues, thus enabling visualization of spatial relationships between surgical instruments and subsurface structures before incisions begin. Preoperative images are generally registered to soft tissues through aligning segmented volumetric image data with an intraoperatively sensed cloud of organ surface points. However, there is currently no viable noncontact minimally invasive scanning technology that can collect these points through a single laparoscopic port, which limits wider adoption of soft-tissue image guidance. In this paper, we describe a system based on conoscopic holography that is capable of minimally invasive surface scanning. We present the results of several validation experiments scanning ex vivo biological and phantom tissues with a system consisting of a tracked, off-the-shelf, relatively inexpensive conoscopic holography unit. These experiments indicate that conoscopic holography is suitable for use with biological tissues, and can provide surface scans of comparable quality to existing clinically used laser range scanning systems that require open surgery. We demonstrate experimentally that conoscopic holography can be used to guide a surgical needle to desired subsurface targets with an average tip error of less than 3 mm.

A 3-D augmented reality navigation system using autostereoscopic images was developed for MRI-guided surgery. The 3-D images are created by employing an animated autostereoscopic image, integral videography (IV), which provides geometrically accurate 3-D spatial images and reproduces motion parallax without using any supplementary eyeglasses or tracking devices. The spatially projected 3-D images are superimposed onto the surgical area and viewed via a half-silvered mirror. A fast and accurate spatial image registration method was developed for intraoperative i.v. image-guided therapy. Preliminary experiments showed that the total system error in patient-to-image registration was 0.90 +/- 0.21 mm, and the procedure time for guiding a needle toward a target was shortened by 75%. An animal experiment was also conducted to evaluate the performance of the system. The feasibility studies showed that augmented reality of the image overlay system could increase the surgical instrument placement accuracy and reduce the procedure time as a result of intuitive 3-D viewing.

This paper studies a family of distributions constructed from multivariate gamma distributions to model the statistical properties of multisensor synthetic aperture radar (SAR) images. These distributions referred to as multisensor multivariate gamma distributions (MuMGDs) are potentially interesting for detecting changes in SAR images acquired by different sensors having different numbers of looks. The first part of this paper compares different estimators for the parameters of MuMGDs. These estimators are based on the maximum likelihood principle, the method of inference function for margins, and the method of moments. The second part of the paper studies change detection algorithms based on the estimated correlation coefficient of MuMGDs. Simulation results conducted on synthetic and real data illustrate the performance of these change detectors.

Large core needle biopsy is a common procedure used to obtain histological samples when cancer is suspected in diagnostic breast images. The procedure is typically performed under image guidance, with freehand ultrasound and stereotactic mammography (SM) being the most common modalities used. To utilize the advantages of both modalities, a biopsy device combining three-dimensional ultrasound (3DUS) and digital SM imaging with computer-aided needle guidance was developed. An implementation of a stereo camera method was applied to SM calibration, providing a target localization error of 0.35 mm. The 3-D transformation between the two imaging modalities was then derived, with a target registration error of 0.52 mm. Finally, the needle guidance error of the device was evaluated using tissue-mimicking phantoms, showing a sample mean and standard deviation of 0.44 +/- 0.22 and 0.49 +/- 0.27 mm for targets planned from 3DUS and SM images, respectively. These results suggest that a biopsy procedure guided using this device would successfully sample breast lesions at a size greater than or equal to the smallest typically detected in mammographic screening (approximately 2 mm).

Simultaneous capturing of ultrasound (US) and magnetic resonance (MR) images allows fusion of information obtained from both modalities. We propose an MR-compatible US system where MR images are acquired in a known orientation with respect to the US imaging plane and concurrent real-time imaging can be achieved. Compatibility of the two imaging devices is a major issue in the physical setup. Tests were performed to quantify the radio frequency (RF) noise introduced in MR and US images, with the US system used in conjunction with MRI scanner of different field strengths (0.5 T and 3 T). Furthermore, simultaneous imaging was performed on a dual modality breast phantom in the 0.5 T open bore and 3 T close bore MRI systems to aid needle-guided breast biopsy. Fiducial based passive tracking and electromagnetic based active tracking were used in 3 T and 0.5 T, respectively, to establish the location and orientation of the US probe inside the magnet bore. Our results indicate that simultaneous US and MR imaging are feasible with properly-designed shielding, resulting in negligible broadband noise and minimal periodic RF noise in both modalities. US can be used for real time display of the needle trajectory, while MRI can be used to confirm needle placement.

Multipinhole single photon emission computed tomography (SPECT) imaging has several advantages over single pinhole SPECT imaging, including an increased sensitivity and an improved sampling. However, the quest for a good design is challenging, due to the large number of design parameters. The effect of one of these, the amount of overlap in the projection images, on the reconstruction image quality, is examined in this paper. The evaluation of the quality is based on efficient approximations for the linearized local impulse response and the covariance in a voxel, and on the bias of the reconstruction of the noiseless projection data. Two methods are proposed that remove the overlap in the projection image by blocking certain projection rays with the use of extra shielding between the pinhole plate and the detector. Also two measures to quantify the amount of overlap are suggested. First, the approximate method, predicting the contrast-to-noise ratio (CNR), is validated using postsmoothed maximum likelihood expectation maximization (MLEM) reconstructions with an imposed target resolution. Second, designs with different amounts of overlap are evaluated to study the effect of multiplexing. In addition, the CNR of each pinhole design is also compared with that of the same design where overlap is removed. Third, the results are interpreted with the overlap quantification measures. Fourth, the two proposed overlap removal methods are compared. From the results we can conclude that, once the complete detector area has been used, the extra sensitivity due to multiplexing is only able to compensate for the loss of information, not to improve the CNR. Removing the overlap, however, improves the CNR. The gain is most prominent in the central field of view, though often at the cost of the CNR of some voxels at the edges, since after overlap removal very little information is left for their reconstruction. The reconstruction images provide insight in the multiplexing and truncation artifacts.

Multimodality medical imaging takes advantage of the strengths of different imaging modalities to provide a more complete picture of the anatomy under investigation. Many complementary modalities have been combined to form such systems and some are gaining use clinically. One combination that has not been developed, in large part due to technical difficulties, is a combined magnetic resonance (MR) and ultrasound (US) imaging system. Such a system offers the potential to combine the strengths of these modalities in a wide range of diagnostic and therapeutic applications. The goal of this study was to evaluate the feasibility of performing simultaneous multimodality US and MR imaging. An US imaging system capable of operation in a clinical MR imager was developed, and methods to perform simultaneous imaging were investigated. Simultaneous imaging was feasible without any mutual interference by either filtering the transmitted and received US signal, or by synchronizing data acquisition between the two imaging systems. Spatial registration between the two modalities was achieved by using a reference phantom with implanted glass beads in orthogonal planes. Excellent agreement was observed between spatial measurements of an object made with both modalities, and the feasibility of using this system in vivo was demonstrated in a rabbit model. Simultaneous US and MR imaging is achievable, and can provide complementary information about an object under investigation. This demonstration of technical feasibility and the development of a prototype system open up the potential to investigate the promising clinical applications of this combined technology.

A fast, robust, accurate, and automatic registration technique based on magnetic resonance (MR) active microcoils (active markers) for registration of tracked medical devices to preprocedural MR-images is presented. This allows for a straight-forward integration of position measurement systems into clinical procedures. The presented method is useful for guidance purposes in clinical applications with high demands on accuracy and ease-of-use (e.g., neurosurgical or orthopedic applications). The determination of the positions of the active markers is integrated into the preparation phase of the actual MR imaging scan. The technique features a generic interface using DICOM standards for communication with navigation workstations linked to an MR system. The position of the active markers is fixed with respect to a reference system of an optical positioning measurement system (OPMS) and thus the coregistration of the MR system and the OPMS is established. In a phantom study, a mean overall targeting accuracy of 0.9+/-0.1 mm was achieved and compared favorably to results obtained from manual registration tests (1.8+/-0.3 mm) carried out in parallel. For a test person trained for both registration methods, workflow improvements of 3-6 min per registration step were found. The need for manual interaction is entirely eliminated thus avoiding user-bias, which is advantageous for the usage in clinical routine. The method improves the ease-of-use of tracking equipment during stereotactic guidance. The method is finally demonstrated in a volunteer study using a model of a Mayfield skull clamp with integrated active and optical reference markers.

The hybrid magnetic resonance (MR)/X-ray suite (XMR) is a recently introduced imaging solution that provides new possibilities for guidance of cardiovascular catheterization procedures. We have previously described and validated a technique based on optical tracking to register MR and X-ray images obtained from the sliding table XMR configuration. The aim of our recent work was to extend our technique by providing an improved calibration stage, real-time guidance during cardiovascular catheterization procedures, and further off-line analysis for mapping cardiac electrical data to patient anatomy. Specially designed optical trackers and a dedicated calibration object have resulted in a single calibration step that can be efficiently checked and updated before each procedure. An X-ray distortion model has been implemented that allows for distortion correction for arbitrary c-arm orientations. During procedures, the guidance system provides a real-time combined MR/X-ray image display consisting of live X-ray images with registered recently acquired MR derived anatomy. It is also possible to reconstruct the location of catheters seen during X-ray imaging in the MR derived patient anatomy. We have applied our registration technique to 13 cardiovascular catheterization procedures. Our system has been used for the real-time guidance of ten radiofrequency ablations and one aortic stent implantation. We demonstrate the real-time guidance using two exemplar cases. In a further two cases we show how off-line analysis of registered image data, acquired during electrophysiology study procedures, has been used to map cardiac electrical measurements to patient anatomy for two different types of mapping catheters. The cardiologists that have used the guidance system suggest that real-time XMR guidance could have substantial value in difficult interventional and electrophysiological procedures, potentially reducing procedure time and delivered radiation dose. Also, the ability to map measured electrical data to patient specific anatomy provides improved visualization and a path to investigation of cardiac electromechanical models.

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patient's body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.

Mutual information-based image registration, shown to be effective in registering a range of medical images, is a computationally expensive process, with a typical execution time on the order of minutes on a modern single-processor computer. Accelerated execution of this process promises to enhance efficiency and therefore promote routine use of image registration clinically. This paper presents details of a hardware architecture for real-time three-dimensional (3-D) image registration. Real-time performance can be achieved by setting up a network of processing units, each with three independent memory buses: one each for the two image memories and one for the mutual histogram memory. Memory access parallelization and pipelining, by design, allow each processing unit to be 25 times faster than a processor with the same bus speed, when calculating mutual information using partial volume interpolation. Our architecture provides superior per-processor performance at a lower cost compared to a parallel supercomputer.

We describe a registration and tracking technique to integrate cardiac X-ray images and cardiac magnetic resonance (MR) images acquired from a combined X-ray and MR interventional suite (XMR). Optical tracking is used to determine the transformation matrices relating MR image coordinates and X-ray image coordinates. Calibration of X-ray projection geometry and tracking of the X-ray C-arm and table enable three-dimensional (3-D) reconstruction of vessel centerlines and catheters from bi-plane X-ray views. We can, therefore, combine single X-ray projection images with registered projection MR images from a volume acquisition, and we can also display 3-D reconstructions of catheters within a 3-D or multi-slice MR volume. Registration errors were assessed using phantom experiments. Errors in the combined projection images (two-dimensional target registration error--TRE) were found to be 2.4 to 4.2 mm, and the errors in the integrated volume representation (3-D TRE) were found to be 4.6 to 5.1 mm. These errors are clinically acceptable for alignment of images of the great vessels and the chambers of the heart. Results are shown for two patients. The first involves overlay of a catheter used for invasive pressure measurements on an MR volume that provides anatomical context. The second involves overlay of invasive electrode catheters (including a basket catheter) on a tagged MR volume in order to relate electrophysiology to myocardial motion in a patient with an arrhythmia. Visual assessment of these results suggests the errors were of a similar magnitude to those obtained in the phantom measurements.

System design and initial phantom accuracy results for a novel biopsy system integrating both magnetic resonance (MR) and ultrasound (US) imaging modalities are presented. A phantom experiment was performed to investigate the efficacy of this hybrid guidance biopsy technique in a breast tissue mimicking phantom. A comparison between MR-guided core biopsy verses MR/US-guided core biopsy of phantom targets was realized using a scoring system based on the consistency of the acquired core samples (14 gauge). It was determined that the addition of US to guide needle placement improved the accuracy from an average score of 7.4 out of 10 (MRI guidance alone), to 9.6 (MRI/US guidance) over 21 trials. The average amount of needle tip correction resulting from the additional US information was determined to be 3.7 mm. This correction value is substantial, equal to approximately one radius of the intended targets. Hybrid US/MRI guided biopsy appears to offer a simple means to ensure accurate breast tissue sampling without the need for repeat MRI scans for verification or the need for real-time imaging in open MRI geometries.

This paper presents a new method for image-guided surgery called image-enhanced endoscopy. Registered real and virtual endoscopic images (perspective volume renderings generated from the same view as the endoscope camera using a preoperative image) are displayed simultaneously; when combined with the ability to vary tissue transparency in the virtual images, this provides surgeons with the ability to see beyond visible surfaces and, thus, provides additional exposure during surgery. A mount with four photoreflective spheres is rigidly attached to the endoscope and its position and orientation is tracked using an optical position sensor. Generation of virtual images that are accurately registered to the real endoscopic images requires calibration of the tracked endoscope. The calibration process determines intrinsic parameters (that represent the projection of three-dimensional points onto the two-dimensional endoscope camera imaging plane) and extrinsic parameters (that represent the transformation from the coordinate system of the tracker mount attached to the endoscope to the coordinate system of the endoscope camera), and determines radial lens distortion. The calibration routine is fast, automatic, accurate and reliable, and is insensitive to rotational orientation of the endoscope. The routine automatically detects, localizes, and identifies dots in a video image snapshot of the calibration target grid and determines the calibration parameters from the sets of known physical coordinates and localized image coordinates of the target grid dots. Using nonlinear lens-distortion correction, which can be performed at real-time rates (30 frames per second), the mean projection error is less than 0.5 mm at distances up to 25 mm from the endoscope tip, and less than 1.0 mm up to 45 mm. Experimental measurements and point-based registration error theory show that the tracking error is about 0.5-0.7 mm at the tip of the endoscope and less than 0.9 mm for all points in the field of view of the endoscope camera at a distance of up to 65 mm from the tip. It is probable that much of the projection error is due to endoscope tracking error rather than calibration error. Two examples of clinical applications are presented to illustrate the usefulness of image-enhanced endoscopy. This method is a useful addition to conventional image-guidance systems, which generally show only the position of the tip (and sometimes the orientation) of a surgical instrument or probe on reformatted image slices.

This paper reports on technology developments aimed at improving the state of the art for image-guided minimally invasive spine procedures. Back pain is a major health problem with serious economic consequences. Minimally invasive procedures to treat back pain are rapidly growing in popularity due to improvements in technique and the substantially reduced trauma to the patient versus open spinal surgery. Image guidance is an enabling technology for minimally invasive procedures, but technical problems remain that may limit the wider applicability of these techniques. The paper begins with a discussion of low back pain and the potential shortcomings of open back surgery. The advantages of minimally invasive procedures are enumerated, followed by a list of technical problems that must be overcome to enable the more widespread dissemination of these techniques. The technical problems include improved intraoperative imaging, fusion of images from multiple modalities, the visualization of oblique paths, percutaneous spine tracking, mechanical instrument guidance, and software architectures for technology integration. Technical developments to address some of these problems are discussed next. The discussion includes intraoperative computerized tomography (CT) imaging, magnetic resonance imaging (MRI)/CT image registration, three-dimensional (3-D) visualization, optical localization, and robotics for percutaneous instrument placement. Finally, the paper concludes by presenting several representative clinical applications: biopsy, vertebroplasty, nerve and facet blocks, and shunt placement. The program presented here is a first step to developing the physician-assist systems of the future, which will incorporate visualization, tracking, and robotics to enable the precision placement and manipulation of instruments with minimal trauma to the patient.