Search

The use of intelligent feedback modalities to control and react to interaction forces during surgical procedures is an important factor in enabling safe and precise surgery. We explore the use of a model-mediated telemanipulation framework to enhance a user’s situational awareness using assistive virtual fixtures and semi-automated task execution for safe and intuitive environment interaction during robotic laparoscopic surgery. The framework allows stiffness mapping with semi-autonomous excitation, hybrid position-force control, and model updates during soft geometry contact. A 24-person study was carried out at 3 sites in simulated ablation and palpation of phantom anatomy. Compared to methods lacking intelligent feedback and guidance, the proposed framework improved task execution metrics (force regulation, completion time, path-following error) and reduced user effort.

Background It was suggested that the lack of haptic feedback, formerly considered a limitation for the da Vinci robotic system, does not affect robotic surgeons because of training and compensation based on visual feedback. However, conclusive studies are still missing, and the interest in force reflection is rising again. Methods We integrated a seven-DoF master into the da Vinci Research Kit. We designed tissue grasping, palpation, and incision tasks with robotic surgeons, to be performed by three groups of users (expert surgeons, medical residents, and nonsurgeons, five users/group), either with or without haptic feedback. Task-specific quantitative metrics and a questionnaire were used for assessment. Results Force reflection made a statistically significant difference for both palpation (improved inclusion detection rate) and incision (decreased tissue damage). Conclusions Haptic feedback can improve key surgical outcomes for tasks requiring a pronounced cognitive burden for the surgeon, to be possibly negotiated with longer completion times.

Magnetic Resonance Imaging (MRI) offers great potential for planning, guiding, monitoring and controlling interventions. MR arthrography (MRAr) is the imaging gold standard for assessing small ligament and fibrocartilage injury in joints. In contemporary practice, MRAr consists of two consecutive sessions: (1) an interventional session where a needle is driven to the joint space and MR contrast is injected under fluoroscopy or CT guidance; and (2) a diagnostic MRI imaging session to visualize the distribution of contrast inside the joint space and evaluate the condition of the joint. Our approach to MRAr is to eliminate the separate radiologically guided needle insertion and contrast injection procedure by performing those tasks on conventional high-field closed MRI scanners. We propose a 2D augmented reality image overlay device to guide needle insertion procedures. This approach makes diagnostic high-field magnets available for interventions without a complex and expensive engineering entourage. In preclinical trials, needle insertions have been performed in the joints of porcine and human cadavers using MR image overlay guidance; in all cases, insertions successfully reached the joint space on the first attempt.

This paper presents the development and evaluation of video augmentation on the stereoscopic da Vinci S system with intraoperative image guidance for base of tongue tumor resection in transoral robotic surgery (TORS). Proposed workflow for image-guided TORS begins by identifying and segmenting critical oropharyngeal structures (e.g., the tumor and adjacent arteries and nerves) from preoperative computed tomography (CT) and/or magnetic resonance (MR) imaging. These preoperative planned data can be deformably registered to the intraoperative endoscopic view using mobile C-arm cone-beam computed tomography (CBCT) [1, 2]. Augmentation of TORS endoscopic video defining surgical targets and critical structures has the potential to improve navigation, spatial orientation, and confidence in tumor resection. Experiments in animal specimens achieved statistically significant improvement in target localization error when comparing the proposed image guidance system to simulated current practice.

We present an image overlay system to aid needle insertion procedures in computed tomography (CT) scanners. The device consists of a display and a semitransparent mirror that is mounted on the gantry. Looking at the patient through the mirror, the CT image appears to be floating inside the patient with correct size and position, thereby providing the physician with two-dimensional (2-D) "X-ray vision" to guide needle insertions. The physician inserts the needle following the optimal path identified in the CT image rendered on the display and, thus, reflected in the mirror. The system promises to reduce X-ray dose, patient discomfort, and procedure time by significantly reducing faulty insertion attempts. It may also increase needle placement accuracy. We report the design and implementation of the image overlay system followed by the results of phantom and cadaver experiments in several clinical applications.

We present a 2D image overlay device to assist needle placement on computed tomography (CT) scanners.The system consists of a flat display and a semitransparent mirror mounted on the gantry. When the physician looks at the patient through the mirror, the CT image appears to be floating inside the body with correct size and position as if the physician had 2D 'X-ray vision'. The physician draws the optimal path on the CT image. The composite image is rendered on the display and thus reflected in the mirror. The reflected image is used to guide the physician in the procedure. In this article, we describe the design and various embodiments of the 2D image overlay system, followed by the results of phantom and cadaver experiments in multiple clinical applications.Multiple skeletal targets were successfully accessed with one insertion attempt. Generally, successful access was recorded on liver targets when a clear path opened, but the number of attempts and accuracy showed variability because of occasional lack of access. Soft tissue deformation further reduced the accuracy and consistency in comparison to skeletal targets.The system demonstrated strong potential for reducing faulty needle insertion attempts, thereby reducing X-ray dose and patient discomfort.

Purpose: Brachytherapy is a standard option of care for prostate cancer patients but may be improved by dynamic dose calculation based on localized seed positions. The American Brachytherapy Society states that the major current limitation of intraoperative treatment planning is the inability to localize the seeds in relation to the prostate. An image-guidance system was therefore developed to localize seeds for dynamic dose calculation. Methods: The proposed system is based on transrectal ultrasound (TRUS) and mobile C-arm fluoroscopy, while using a simple fiducial with seed-like markers to compute pose from the nonencoded C-arm. Three or more fluoroscopic images and an ultrasound volume are acquired and processed by a pipeline of algorithms: (1) seed segmentation, (2) fiducial detection with pose estimation, (3) seed matching with reconstruction, and (4) fluoroscopy-to-TRUS registration. Results: The system was evaluated on ten phantom cases, resulting in an overall mean error of 1.3 mm. The system was also tested on 37 patients and each algorithm was evaluated. Seed segmentation resulted in a 1% false negative rate and 2% false positive rate. Fiducial detection with pose estimation resulted in a 98% detection rate. Seed matching with reconstruction had a mean error of 0.4 mm. Fluoroscopy-to-TRUS registration had a mean error of 1.3 mm. Moreover, a comparison of dose calculations between the authors’ intraoperative method and an independent postoperative method shows a small difference of 7% and 2% forD 90 and V 100, respectively. Finally, the system demonstrated the ability to detect cold spots and required a total processing time of approximately 1 min. Conclusions: The proposed image-guidance system is the first practical approach to dynamic dose calculation, outperforming earlier solutions in terms of robustness, ease of use, and functional completeness.

Low-dose-rate prostate brachytherapy is a treatment option for low- and mid-risk prostate cancer through introduction of radioactive seeds into the prostate. Seed placement deviations are common and associated with postoperative complications. Dynamic dosimetry is a method to accurately localize the true position of the seeds inside the tissue, calculate the delivered dose, and adapt the implant plan accordingly to compensate for seed placement deviations in the operating room. A practical method for dynamic dosimetry relies on localization of the implanted seeds in 3D space from several C-arm images and registering them to a 3D ultrasound volume of the prostate region. In this chapter we introduce a system and workflow for intraoperative dosimetry for prostate brachytherapy. In the suggested workflow, C-arm images are acquired from different angles and are used to reconstruct the seeds in 3D space. For this purpose, we rely on a method based on dimensionality reduced linear programming to match the projections of a seed in different images and localize the seed positions after automatic C-arm pose correction. In the next step of the workflow, the reconstructed seeds are registered to an ultrasound volume of the prostate in a point-to-volume registration scheme. We tested our method on data from 16 patients and compared our dosimetry results with results from Day-1 CT. In comparison, we achieved absolute error of 2.2 ± 1.8 % (mean ± STD) in estimating the percentage of the prostate volume that receives 100 % of the prescribed dose (V100) and absolute error of 10.5 ± 9.5 % in prediction of the minimum dose delivered to 90 % of the prostate (D90).

The lack of dynamic dosimetry tools for permanent prostate brachytherapy causes otherwise avoidable problems in prostate cancer patient care. The goal of this work is to satisfy this need in a readily adoptable manner. Using the ubiquitous ultrasound scanner and mobile non-isocentric C-arm, we show that dynamic dosimetry is now possible with only the addition of an arbitrarily configured marker-based fiducial. Not only is the system easily configured from accessible hardware, but it is also simple and convenient, requiring little training from technicians. Furthermore, the proposed system is built upon robust algorithms of seed segmentation, fiducial detection, seed reconstruction, and image registration. All individual steps of the pipeline have been thoroughly tested, and the system as a whole has been validated on a study of 25 patients. The system has shown excellent results of accurately computing dose, and does so with minimal manual intervention, therefore showing promise for widespread adoption of dynamic dosimetry.

C-arm X-ray fluoroscopy-based radioactive seed localization for intraoperative dosimetry of prostate brachytherapy is an active area of research. The fluoroscopy tracking (FTRAC) fiducial is an image-based tracking device composed of radio-opaque BBs, lines, and ellipses that provides an effective means for pose estimation so that three-dimensional reconstruction of the implanted seeds from multiple X-ray images can be related to the ultrasound-computed prostate volume. Both the FTRAC features and the brachytherapy seeds must be segmented quickly and accurately during the surgery, but current segmentation algorithms are inhibitory in the operating room (OR). The first reason is that current algorithms require operators to manually select a region of interest (ROI), preventing automatic pipelining from image acquisition to seed reconstruction. Secondly, these algorithms fail often, requiring operators to manually correct the errors. We propose a fast and effective ROI-free automatic FTRAC and seed segmentation algorithm to minimize such human intervention. The proposed algorithm exploits recent image processing tools to make seed reconstruction as easy and convenient as possible. Preliminary results on 162 patient images show this algorithm to be fast, effective, and accurate for all features to be segmented. With near perfect success rates and subpixel differences to manual segmentation, our automatic FTRAC and seed segmentation algorithm shows promising results to save crucial time in the OR while reducing errors.

Prostate brachytherapy is a treatment for prostate cancer using radioactive seeds that are permanently implanted in the prostate. The treatment success depends on adequate coverage of the target gland with a therapeutic dose, while sparing the surrounding tissue. Since seed implantation is performed under transrectal ultrasound (TRUS) imaging, intraoperative localization of the seeds in ultrasound can provide physicians with dynamic dose assessment and plan modification. However, since all seeds cannot be seen in the ultrasound images, registration between ultrasound and fluoroscopy is a practical solution for intraoperative dosimetry. In this manuscript, we introduce a new image-based nonrigid registration method that obviates the need for manual seed segmentation in TRUS images and compensates for the prostate displacement and deformation due to TRUS probe pressure. First, we filter the ultrasound images for subsequent registration using thresholding and Gaussian blurring. Second, a computationally efficient point-to-volume similarity metric, an affine transformation and an evolutionary optimizer are used in the registration loop. A phantom study showed final registration errors of 0.84 ± 0.45 mm compared to ground truth. In a study on data from 10 patients, the registration algorithm showed overall seed-to-seed errors of 1.7 ± 1.0 mm and 1.5 ± 0.9 mm for rigid and nonrigid registration methods, respectively, performed in approximately 30 seconds per patient.

The success of prostate brachytherapy critically depends on delivering adequate dose to the prostate gland, and the capability of intraoperatively localizing implanted seeds provides potential for dose evaluation and optimization during therapy. REDMAPS is a recently reported algorithm that carries out seed localization by detecting, matching and reconstructing seeds in only a few seconds from three acquired x-ray images (Lee et al 2011 IEEE Trans. Med. Imaging 29 38–51). In this paper, we present an automatic pose correction (APC) process that is combined with REDMAPS to allow for both more accurate seed reconstruction and the use of images with relatively large pose errors. APC uses a set of reconstructed seeds as a fiducial and corrects the image pose by minimizing the overall projection error. The seed matching and APC are iteratively computed until a stopping condition is met. Simulations and clinical studies show that APC significantly improves the reconstructions with an overall average matching rate of ≥99.4%, reconstruction error of ≤0.5 mm, and the matching solution optimality of ≥99.8%.

Ultrasound-Fluoroscopy fusion is a key step toward intraoperative dosimetry for prostate brachytherapy. We propose a method for intensity-based registration of fluoroscopy to ultrasound that obviates the need for seed segmentation required for seed-based registration. We employ image thresholding and morphological and Gaussian filtering to enhance the image intensity distribution of ultrasound volume. Finally, we find the registration parameters by maximizing a point-to-volume similarity metric. We conducted an experiment on a ground truth phantom and achieved registration error of 0.7 +/- 0.2 mm. Our clinical results on 5 patient data sets show excellent visual agreement between the registered seeds and the ultrasound volume with a seed-to-seed registration error of 1.8 +/- 0.9mm. With low registration error, high computational speed and no need for manual seed segmentation, our method is promising for clinical application.

Purpose To evaluate a prototypical system of dynamic intraoperative dosimetry for prostate brachytherapy using registered ultrasound and fluoroscopy (RUF) with a nonisocentric C-arm (GE OEC, Salt Lake City, UT) and to compare intraoperative dosimetry of RUF as well as ultrasound-based seed localization (USD) with Day 0 CT dosimetry. Methods Seed positions were independently determined using RUF and USD. RUF uses a radio-opaque fiducial for registration to ultrasound and 3-dimensional reconstruction of seeds relative to prostate using nonisocentric C-arm fluoroscopy. Postimplant CT was performed on Day 0. Squared differences between dosimetric measures for RUF vs. CT and USD vs. CT were calculated and mean squared differences evaluated. Paired t test was used to evaluate which method was more closely aligned with CT. Accuracies of USD and RUF compared with CT were estimated using a nonparametric approach. Results Six patients were treated and compared with USD. RUF identified areas of underdosage intraoperatively in all patients and median 5 additional seeds were placed. In 40 of 42 measures, RUF was equally or more closely correlated with CT than USD. USD showed statistically significant variation from CT for 6 of 7 parameters compared with 1 of 7 parameters for RUF. Mean squared differences from CT were significantly smaller for RUF in 4 of 7 parameters compared with USD. Conclusions Dynamic intraoperative dosimetry is possible with a conventional nonisocentric C-arm. Compared with an USD method, RUF-based intraoperative dosimetry was more closely aligned with immediate postimplant CT. RUF identified areas of underdosage, which were not detected using USD.

Purpose: Prostate brachytherapy is a common treatment method for low-risk prostate cancer patients. Intraoperative treatment planning is known to improve the treatment procedure and the outcome. The current limitation of intraoperative treatment planning is the inability to localize the seeds in relation to the prostate. We developed an image-guidance system to fulfill this need to achieve intraoperative dynamic dosimetry in prostate brachytherapy. Methods: Our system is based on standard imaging equipments available in the operating room, including the transrectal ultrasound (TRUS) and the mobile C-arm. A simple fiducial is added to compute the C-arm pose. Three fluoroscopic images and an ultrasound volume of the seeds and the prostate are acquired and processed by four image processing algorithms: seed segmentation, fiducial detection with pose estimation, seed reconstruction, and seeds-to-TRUS registration. The updated seed positions allow the physician to assess the quality of implantation and dynamically adjust the treatment plan during the course of surgery to achieve improved exit dosimetry. Results: The system was tested on 10 phantoms and 37 patients. Seed segmentation resulted in a 1% false negative and 2% false positive rates. Fiducial detection with pose estimation resulted in a detection rate of 98%. Seed reconstruction had a mean reconstructionmore » error of 0.4 mm. Seeds-to-TRUS registration had a mean registration error of 1.3 mm. The total processing time from image acquisition to registration was approximately 1 minute. Conclusion: We present an image-guidance system for intraoperative dynamic dosimetry in prostate brachytherapy. Using standard imaging equipments and a simple fiducial, our system can be easily adopted in any clinics. Robust image processing algorithms enable accurate and fast computation of the delivered dose. Especially, the system enables detection of possible hot/cold spots during the surgery, allowing the physician to address these in the operating room rather than requiring additional therapy afterwards. This research was supported in part by the National Institutes of Health/National Cancer Institute (NIH/NCI) under grant 2R44CA099374 and grant 1R01CA151395, and in part by the Department of Defense (DoD) under grant W81XWH-05-1-0407.« less

Purpose/Objective(s): The ability to more accurately determine dosimetry intraoperatively during permanent prostate brachytherapy may assist clinicians in improving dosimetric and clinical results. Our purpose was to compare a novel system of registered ultrasound and fluoroscopy (RUF) with the current method of ultrasound (US) based seed localization (a.k.a. real time) for intraoperatively predicting dosimetry, using post-implant CT as a standard.

Purpose: Intra‐operative dosimetryoptimization of TRUS‐guided prostate brachytherapy requires localization of seeds relative to the prostate [Todor et al. PMB 48(9):1153–71]. Method and Materials: Seeds were reconstructed from C‐arm fluoroscopy, registered to TRUS, and exported to commercial brachytherapy system for dosimetryoptimization. Technical obstacles included pose‐dependant C‐arm calibration; distortion correction; pose estimation of C‐arm images; registering C‐arm to TRUS; and seed reconstruction. We proved that calibration is not critical and distortion correction in AP suffices. A radiographic tracking fiducial was attached in known pose over the template to recover the C‐arm pose from a single image, relative to the template. The 3D coordinates of the segmented seeds are calculated upon resolving the correspondence of seeds in the multiple C‐arm images, by formalizing seed‐matching as a network‐flow problem [Jain et al. Med Phys, 32(11):3475–92]. The seed locations are exported in template coordinates to the Interplant® commercial system for dosimetry analysis and optimization.Results: In precision‐machined hard phantoms with 40–100 seeds, we correctly reconstructed 98.5% seeds with a mean 3D accuracy of 0.63mm (0.91mm error for mismatched seeds). In soft tissue phantoms with 45–87 seeds and clinically realistic 15° C‐arm motion, we correctly reconstructed 100% seeds with 1.5mm absolute accuracy (0.25mm relative accuracy), and registered them to the prostate segmented from TRUS with an accuracy of 3.4mm (0.82mm relative). In a Phase‐1 clinical trial, so far on 4 patients with 66–84 seeds, we achieved intra‐operative monitoring of seed distribution and dosimetry. We optimized the V100 dose by inserting 3–9 additional seeds. Conclusion: Intra‐operative doseoptimization is possible with an average C‐arm, at negligible additional cost to existing clinical installations. Conflict of Interest: The work has been done in collaboration with Acoustic MedSystems for seed import into the Interplant®. Keywords: Prostate Brachytherapy, Intra‐operative Dosimetry.Support: DoD/PC‐050170, DoD/PC‐050042, NIH‐1R43CA099374, NSF‐9731478.

Purpose: Intra‐operative dosimetry in prostate brachytherapy critically depends on discerning the 3‐D locations of implanted seeds. The accuracy of 3‐D seed reconstruction step is, in turn, limited by the accuracy with which the position and orientation of individual implanted seed in the fluoroscopic images can be found. A method for robustly segmenting the seeds in fluoroscopic images is proposed here. Methods and Materials: The process of determining the locations and orientations of implanted seeds is sub‐divided into three main steps. In the first step, the image is segmented by shape‐size based morphological approach to eliminate background noise and do away with non‐uniform brightness of the image, to get seed‐like regions. These regions are either single seeds or overlapping multiple seed clusters. In the second step, the regions are analyzed and classified definitively, in a two‐phase statistical process coupled with information extraction from original intensity image, into two classes: single seed and overlapping multiple seed cluster. In the third step, the region belonging to overlapping multiple seed cluster is resolved into its constituent individual seeds through a simple and novel technique. Results: The proposed algorithm was tested on a set of ten clinical fluoroscopic images. The algorithm correctly determines the seeds with overall average of 99.57%. The clusters are not correctly resolved only in two images (2 clusters each, 1.7% and 1.6% of total seeds in respective implants). One false positive (noise labeled as seed) each is reported in two images, both the cases being where the tip of catheter appears to be of the size and shape of seed. Conclusions: The algorithm builds on an existing framework of morphological processing and provides further improvements in classification and cluster resolution. The algorithm appears to be robust and accurate despite the poor resolution of clinical images.