Search

With significant progress being made toward improving endoscope technology such as capsule endoscopy and robotic endoscopy, the development of advanced strategies for manipulating, controlling, and more generally, easing the accessibility of these devices for physicians is an important next step. This article presents an autonomous navigation strategy for use in endoscopy, utilizing a state-dependent region estimation approach to allow for multimodal control design. This region estimator is evaluated for its accuracy in predicting yaw angle of the camera relative to the lumen center, and for estimating the location of the camera based on overall haustra morphology within the colon. To assess the utility of this region estimator, multimodal control is used to allow for autonomous navigation of the Endoculus, a robotic capsule endoscope, within a benchtop, to-scale, simulated colon. The estimation approach is presented and tested, demonstrating successful tracking of fixed velocity rotations at speeds up to $40^\circ$ /s and allowing for curve anticipation approximately 10 cm before entering a curved section of the simulator. Finally, the multimodal control strategy utilizing this estimator is tested within the simulator over a variety of anatomic configurations. This strategy proves successful for navigation in both straight sections of this simulator and in tightly curved sections as small as 8 cm radius of curvature, with average velocities reaching 2.61 cm/s in straight sections and 0.99 cm/s in curved sections.

Robotic capsule endoscopes (RCEs) are being widely investigated to improve the state of various endoscopy procedures. This article presents the novel design of a multi-DOF sensor-enabled RCE for colonoscopies (Endoculus) and evaluates porcine in vivo and ex vivo performance. The novelty of the design includes a custom “double-worm” drive that removes axial gear forces while reducing radial moments, and the full parameterization of gear geometries allows for size minimization via an optimization routine over design constraints. Two independently controlled motors drive micro-pillared treads above and below the device allowing for two-degrees of freedom (2-DOF) skid-steering, even in a collapsed lumen. The Endoculus contains all functionality of a traditional endoscope: a camera, adjustable light emitting diodes (LEDs), channels for insufflation and irrigation, and a tool port for endoscopy instruments (e.g., forceps, snares, etc.). Additionally, the Endoculus carries an inertial measurement unit, magnetometer, motor encoders, and motor current sensors to aid in future autonomy strategies. Porcine surgical evaluation demonstrated locomotion up to 40 mm/s on the colon mucosa, 2-DOF steering, the ability to traverse haustral folds, and functionality of endoscopy tools. This platform will enable future validation of feedback control, localization, and mapping algorithms in the unconventional in vivo environment.

While half of all construction tasks can be fully automated the other half relies to a certain degree on human support. This paper presents a Computer Vision (CV) and Human–Robot Interaction/Collaboration (HRI/C) supported Design-to-Robotic-Assembly (D2RA) approach that links computational design with robotic assembly. This multidisciplinary approach has been tested on a case study focusing on urban furniture and involving experts from respective disciplines and students.

Objective: Robotic endoscopes have the potential to dramatically improve endoscopy procedures, however current attempts remain limited due to mobility and sensing challenges and have yet to offer the full capabilities of traditional tools. Endoscopic intervention (e.g., biopsy) for robotic systems remains an understudied problem and must be addressed prior to clinical adoption. This paper presents an autonomous intervention technique onboard a Robotic Endoscope Platform (REP) using endoscopy forceps, an auto-feeding mechanism, and positional feedback. M ethods: A workspace model is established for estimating tool position while a Structure from Motion (SfM) approach is used for target-polyp position estimation with the onboard camera and positional sensor. Utilizing this data, a visual system for controlling the REP position and forceps extension is developed and tested within multiple anatomical environments. Results: The workspace model demonstrates accuracy of 5.5% while the target-polyp estimates are within 5 mm of absolute error. This successful experiment requires only 15 seconds once the polyp has been located, with a success rate of 43% using a 1 cm polyp, 67% for a 2 cm polyp, and 81% for a 3 cm polyp. Conclusion: Workspace modeling and visual sensing techniques allow for autonomous endoscopic intervention and demonstrate the potential for similar strategies to be used onboard mobile robotic endoscopic devices. Significance: To the authors’ knowledge this is the first attempt at automating the task of colonoscopy intervention onboard a mobile robot. While the REP is not sized for actual procedures, these techniques are translatable to devices suitable for in vivo application.

This paper presents a new open-source dataset with ground truth position in a simulated colon environment to promote development of real-time feedback systems for physicians performing colonoscopies. Four systems (DSO, LSD-SLAM, SfMLearner, ORB-SLAM2) are tested on this dataset and their failures are analyzed. A data collection platform was fabricated and used to take the dataset in a colonoscopy training simulator that was affixed to a flat surface. The noise in the ground truth positional data induced from the metal in the data collection platform was then characterized and corrected. The Absolute Trajectory RMSE Error (ATE) and Relative Error (RE) metrics were performed on each of the sequences in the dataset for each of the Simultaneous Localization And Mapping (SLAM) systems. While these systems all had good performance in idealized conditions, more realistic conditions in the harder sequences caused them to produce poor results or fail completely. These failures will be a hindrance to physicians in a real-world scenario, so future systems made for this environment must be more robust to the difficulties found in the colon, even at the expense of trajectory accuracy. The authors believe that this is the first open-source dataset with groundtruth data displaying a simulated in vivo environment with active deformation, and that this is the first step toward achieving useful SLAM within the colon. The dataset is available at www.colorado.edu/lab/amtl/datasets.

A scaled robotic endoscopy platform (REP) was previously developed to efficiently test new control schemes in a simulated colon environment. This article presents the derivation and tuning of a nonlinear model of the REP operating on various substrates. The modeling technique and novel empirical friction profiling demonstrated here are useful for a wide variety of devices interacting with unconventional substrates. The model is first derived from the REP drivetrain inertial characteristics, and then the interaction with synthetic tissue is quantified by an automated traction measurement system for multiple substrates. The resulting model is then used with ground-truth VICON and sensor data to optimize uncertain parameters by minimizing pose error over a variety of tests and substrates. The results show an average error reduction of 67% over all tests and substrates, with a worst-case 10% open-loop final position error. The success of these results proves a robust dynamic model of the REP and its tissue interactions without the need to model complex and computationally expensive viscoelastic material properties or discrete/nonlinear events such as stalling. The resulting model will be used to develop model-based feedback control for estimation, disturbance rejection, and autonomy for the REP in an actuated colon simulator.

While autonomous surgical robotic systems exist primarily at the research level, recently these systems have made a strong push into clinical settings. The autonomous or semi-autonomous control of surgical robotic platforms may offer significant improvements to a diverse field of surgical procedures, allowing for high precision, intelligent manipulation of these systems and opening the door to advanced minimally invasive surgical procedures not currently possible.This review highlights those experimental systems currently under development with a focus on in vivo modeling and control strategies designed specifically for the complex and dynamic surgical environment. Expert review: Novel methods for state estimation, system modeling and disturbance rejection, as applied to these devices, continues to improve the performance of these important surgical tools. Procedures such as Natural Orifice Transluminal Endoscopic Surgery and Laparo-Endoscopic Single Site surgery, as well as more conventional procedures such as Colonoscopy, serve to benefit tremendously from the development of these automated robotic systems, enabling surgeons to minimize tissue damage and shorten procedure times while avoiding the consequences of laparotomy.

Capsule endoscopes have gained popularity over the last decade as minimally invasive devices for diagnosing gastrointestinal abnormalities such as colorectal cancer. While this technology offers a less invasive and more convenient alternative to traditional scopes, these capsules are only able to provide observational capabilities due to their passive nature. With the addition of a reliable mobility system and a real-time navigation system, capsule endoscopes could transform from observational devices into active surgical tools, offering biopsy and therapeutic capabilities and even autonomous navigation in a single minimally invasive device. In this work, a vision system is developed to allow for autonomous lumen center tracking and haustral fold identification and tracking during colonoscopy. This system is tested for its ability to accurately identify and track multiple haustral folds across many frames in both simulated and in vivo video, and the lumen center tracking is tested onboard a robotic endoscope platform (REP) within an active simulator to demonstrate autonomous navigation. In addition, real-time localization is demonstrated using open source ORB-SLAM2. The vision system successfully identified 95.6% of Haustral folds in simulator frames and 70.6% in in vivo frames and false positives occurred in less than 1% of frames. The center tracking algorithm showed in vivo center estimates within a mean error of 6.6% of physician estimates and allowed for the REP to traverse 2 m of the active simulator in 6 minutes without intervention.

This work presents an analysis and comparison of the efficacy of two methods for pedicle screw placement during posterior spinal fusion surgery. A total of 100 screws (64 manual and 36 power driven), all placed utilizing a surgical navigation system, were analyzed and compared. Final screw placement was compared to initial surgical plans using the navigation system, and the final screw locations were analyzed on the basis of angular deviation from these planned trajectories as well as screw translation within a critical reference plane. The power driver was found to insignificantly decrease the resulting angular deviation of these pedicle screws with a mean deviation of 3.35 degrees compared to 3.44 degrees with the manual driver (p=0.853). Conversely, the power driver was found to increase the translational distance in the critical region, with mean deviations of 2.45 mm for the power driver compared to 1.54 mm with the manual driver. The increase in translational deviation was significant (p=0.002) indicating that there may be some loss in performance from the adoption of the power driver.