Search

Purpose: In twin-to-twin transfusion syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the two fetuses. In the current practice, TTTS is treated surgically by closing abnormal anastomoses using laser ablation. This surgery is minimally invasive and relies on fetoscopy. Limited field of view makes anastomosis identification a challenging task for the surgeon. Methods: To tackle this challenge, we propose a learning-based framework for in vivo fetoscopy frame registration for field-of-view expansion. The novelties of this framework rely on a learning-based keypoint proposal network and an encoding strategy to filter (i) irrelevant keypoints based on fetoscopic semantic image segmentation and (ii) inconsistent homographies. Results: We validate our framework on a dataset of six intraoperative sequences from six TTTS surgeries from six different women against the most recent state-of-the-art algorithm, which relies on the segmentation of placenta vessels. Conclusion: The proposed framework achieves higher performance compared to the state of the art, paving the way for robust mosaicking to provide surgeons with context awareness during TTTS surgery.

Purpose: Intracardiac transcatheter interventions allow for reducing trauma and hospitalization stays as compared to standard surgery. In the treatment of mitral regurgitation, the most widely adopted transcatheter approach consists in deploying a clip on the mitral valve leaflets by means of a catheter that is run through veins from a peripheral access to the left atrium. However, precise manipulation of the catheter from outside the body while copying with the path constraints imposed by the vessels remains challenging. Methods: We proposed a path tracking control framework that provides adequate motion commands to the robotic steerable catheter for autonomous navigation through vascular lumens. The proposed work implements a catheter kinematic model featuring nonholonomic constraints. Relying on the real-time measurements from an electromagnetic sensor and a fiber Bragg grating sensor, a two-level feedback controller was designed to control the catheter. Results: The proposed method was tested in a patient-specific vessel phantom. A median position error between the center line of the vessel and the catheter tip trajectory was found to be below 2 mm, with a maximum error below 3 mm. Statistical testing confirmed that the performance of the proposed method exhibited no significant difference in both free space and the contact region. Conclusion: The preliminary in vitro studies presented in this paper showed promising accuracy in navigating the catheter within the vessel. The proposed approach enables autonomous control of a steerable catheter for transcatheter cardiology interventions without the request of calibrating the intuitive parameters or acquiring a training dataset.

Purpose:: The integration of a surgical robotic instrument tracking module within optical microscopes holds the potential to advance microsurgery practices, as it facilitates automated camera movements, thereby augmenting the surgeon’s capability in executing surgical procedures. Methods:: In the present work, an innovative detection backbone based on spatial attention module is implemented to enhance the detection accuracy of small objects within the image. Additionally, we have introduced a robust data association technique, capable to re-track surgical instrument, mainly based on the knowledge of the dual-instrument robotics system, Intersection over Union metric and Kalman filter. Results:: The effectiveness of this pipeline was evaluated through testing on a dataset comprising ten manually annotated videos of anastomosis procedures involving either animal or phantom vessels, exploiting the Symani®Surgical System—a dedicated robotic platform designed for microsurgery. The multiple object tracking precision (MOTP) and the multiple object tracking accuracy (MOTA) are used to evaluate the performance of the proposed approach, and a new metric is computed to demonstrate the efficacy in stabilizing the tracking result along the video frames. An average MOTP of 74±0.06% and a MOTA of 99±0.03% over the test videos were found. Conclusion:: These results confirm the potential of the proposed approach in enhancing precision and reliability in microsurgical instrument tracking. Thus, the integration of attention mechanisms and a tailored data association module could be a solid base for automatizing the motion of optical microscopes.

Surgical workflow recognition and context-aware systems could allow better decision making and surgical planning by providing the focused information, which may eventually enhance surgical outcomes. While current developments in computer-assisted surgical systems are mostly focused on recognizing surgical phases, they lack recognition of surgical workflow sequence and other contextual element, e.g., “Instruments.” Our study proposes a hybrid approach, i.e., using deep learning and knowledge representation, to facilitate recognition of the surgical workflow. We implemented “Deep-Onto” network, which is an ensemble of deep learning models and knowledge management tools, ontology and production rules. As a prototypical scenario, we chose robot-assisted partial nephrectomy (RAPN). We annotated RAPN videos with surgical entities, e.g., “Step” and so forth. We performed different experiments, including the inter-subject variability, to recognize surgical steps. The corresponding subsequent steps along with other surgical contexts, i.e., “Actions,” “Phase” and “Instruments,” were also recognized. The system was able to recognize 10 RAPN steps with the prevalence-weighted macro-average (PWMA) recall of 0.83, PWMA precision of 0.74, PWMA F1 score of 0.76, and the accuracy of 74.29% on 9 videos of RAPN. We found that the combined use of deep learning and knowledge representation techniques is a promising approach for the multi-level recognition of RAPN surgical workflow.

Focal epilepsy is a neurological disease that can be surgically treated by removing area of the brain generating the seizures. The stereotactic electroencephalography (SEEG) procedure allows patient brain activity to be recorded in order to localize the onset of seizures through the placement of intracranial electrodes. The planning phase can be cumbersome and very time consuming, and no quantitative information is provided to neurosurgeons regarding the safety and efficacy of their trajectories. In this work, we present a novel architecture specifically designed to ease the SEEG trajectory planning using the 3D Slicer platform as a basis. Trajectories are automatically optimized following criteria like vessel distance and insertion angle. Multi-trajectory optimization and conflict resolution are optimized through a selective brute force approach based on a conflict graph construction. Additionally, electrode-specific optimization constraints can be defined, and an advanced verification module allows neurosurgeons to evaluate the feasibility of the trajectory. A retrospective evaluation was performed using manually planned trajectories on 20 patients: the planning algorithm optimized and improved trajectories in 98% of cases. We were able to resolve and optimize the remaining 2% by applying electrode-specific constraints based on manual planning values. In addition, we found that the global parameters used discards 68% of the manual planned trajectories, even when they represent a safe clinical choice. Our approach improved manual planned trajectories in 98% of cases in terms of quantitative indexes, even when applying more conservative criteria with respect to actual clinical practice. The improved multi-trajectory strategy overcomes the previous work limitations and allows electrode optimization within a tolerable time span.

Single-incision laparoscopic surgery decreases postoperative infections, but introduces limitations in the surgeon’s maneuverability and in the surgical field of view. This work aims at enhancing intra-operative surgical visualization by exploiting the 3D information about the surgical site. An interactive guidance system is proposed wherein the pose of preoperative tissue models is updated online. A critical process involves the intra-operative acquisition of tissue surfaces. It can be achieved using stereoscopic imaging and 3D reconstruction techniques. This work contributes to this process by proposing new methods for improved dense 3D reconstruction of soft tissues, which allows a more accurate deformation identification and facilitates the registration process. Two methods for soft tissue 3D reconstruction are proposed: Method 1follows the traditional approach of the block matching algorithm. Method 2performs a nonparametric modified census transform to be more robust to illumination variation. The simple linear iterative clustering(SLIC) super-pixel algorithm is exploited for disparity refinement by filling holes in the disparity images. The methods were validated using two video datasets from the Hamlyn Centre, achieving an accuracy of 2.95 and 1.66 mm, respectively. A comparison with ground-truth data demonstrated the disparity refinement procedure: (1) increases the number of reconstructed points by up to 43 % and (2) does not affect the accuracy of the 3D reconstructions significantly. Both methods give results that compare favorably with the state-of-the-art methods. The computational time constraints their applicability in real time, but can be greatly improved by using a GPU implementation.