Search

The field of computer vision applied to videos of minimally invasive surgery is ever-growing. Workflow recognition pertains to the automated recognition of various aspects of a surgery: including which surgical steps are performed; and which surgical instruments are used. This information can later be used to assist clinicians when learning the surgery; during live surgery; and when writing operation notes. The Pituitary Vision (PitVis) 2023 Challenge tasks the community to step and instrument recognition in videos of endoscopic pituitary surgery. This is a unique task when compared to other minimally invasive surgeries due to the smaller working space, which limits and distorts vision; and higher frequency of instrument and step switching, which requires more precise model predictions. Participants were provided with 25-videos, with results presented at the MICCAI-2023 conference as part of the Endoscopic Vision 2023 Challenge in Vancouver, Canada, on 08-Oct-2023. There were 18-submissions from 9-teams across 6-countries, using a variety of deep learning models. A commonality between the top performing models was incorporating spatio-temporal and multi-task methods, with greater than 50% and 10% macro-F1-score improvement over purely spacial single-task models in step and instrument recognition respectively. The PitVis-2023 Challenge therefore demonstrates state-of-the-art computer vision models in minimally invasive surgery are transferable to a new dataset, with surgery specific techniques used to enhance performance, progressing the field further. Benchmark results are provided in the paper, and the dataset is publicly available at: https://doi.org/10.5522/04/26531686.

Phase recognition in surgical videos is crucial for enhancing computer-aided surgical systems as it enables automated understanding of sequential procedural stages. Existing methods often rely on fixed temporal windows for video analysis to identify dynamic surgical phases. Thus, they struggle to simultaneously capture short-, mid-, and long-term information necessary to fully understand complex surgical procedures. To address these issues, we propose Multi-Scale Transformers for Surgical Phase Recognition (MuST), a novel Transformer-based approach that combines a Multi-Term Frame encoder with a Temporal Consistency Module to capture information across multiple temporal scales of a surgical video. Our Multi-Term Frame Encoder computes interdependencies across a hierarchy of temporal scales by sampling sequences at increasing strides around the frame of interest. Furthermore, we employ a long-term Transformer encoder over the frame embeddings to further enhance long-term reasoning. MuST achieves higher performance than previous state-of-the-art methods on three different public benchmarks.

Quantum control techniques play an important role in manipulating and harnessing the properties of different quantum systems, including isolated atoms. Here, we propose to achieve quantum control over a single on-surface atomic spin using Landau-Zener-St\"uckelberg-Majorana (LZSM) interferometry implemented with Scanning Tunneling Microscopy (STM). Specifically, we model how the application of time-dependent, non-resonant AC electric fields across the STM tip-surface gap makes it possible to achieve precise quantum state manipulation in an isolated Fe atom on a MgO/Ag(100) surface. We propose a protocol to combine Landau Zener tunneling with LZSM interferometry that permits one to measure the quantum spin tunneling of an individual Fe atom. The proposed experiments can be implemented with ESR-STM instrumentation, opening a new venue in the research of on-surface single spin control., Comment: 6 pages, 4 figures

This paper presents the Holistic and Multi-Granular Surgical Scene Understanding of Prostatectomies (GraSP) dataset, a curated benchmark that models surgical scene understanding as a hierarchy of complementary tasks with varying levels of granularity. Our approach enables a multi-level comprehension of surgical activities, encompassing long-term tasks such as surgical phases and steps recognition and short-term tasks including surgical instrument segmentation and atomic visual actions detection. To exploit our proposed benchmark, we introduce the Transformers for Actions, Phases, Steps, and Instrument Segmentation (TAPIS) model, a general architecture that combines a global video feature extractor with localized region proposals from an instrument segmentation model to tackle the multi-granularity of our benchmark. Through extensive experimentation, we demonstrate the impact of including segmentation annotations in short-term recognition tasks, highlight the varying granularity requirements of each task, and establish TAPIS's superiority over previously proposed baselines and conventional CNN-based models. Additionally, we validate the robustness of our method across multiple public benchmarks, confirming the reliability and applicability of our dataset. This work represents a significant step forward in Endoscopic Vision, offering a novel and comprehensive framework for future research towards a holistic understanding of surgical procedures., Comment: Preprint submitted to Medical Image Analysis. Official extension of previous MICCAI 2022 (https://link.springer.com/chapter/10.1007/978-3-031-16449-1_42) and ISBI 2023 (https://ieeexplore.ieee.org/document/10230819) orals. Data and codes are available at https://github.com/BCV-Uniandes/GraSP

Surgical tool segmentation and action recognition are fundamental building blocks in many computer-assisted intervention applications, ranging from surgical skills assessment to decision support systems. Nowadays, learning-based action recognition and segmentation approaches outperform classical methods, relying, however, on large, annotated datasets. Furthermore, action recognition and tool segmentation algorithms are often trained and make predictions in isolation from each other, without exploiting potential cross-task relationships. With the EndoVis 2022 SAR-RARP50 challenge, we release the first multimodal, publicly available, in-vivo, dataset for surgical action recognition and semantic instrumentation segmentation, containing 50 suturing video segments of Robotic Assisted Radical Prostatectomy (RARP). The aim of the challenge is twofold. First, to enable researchers to leverage the scale of the provided dataset and develop robust and highly accurate single-task action recognition and tool segmentation approaches in the surgical domain. Second, to further explore the potential of multitask-based learning approaches and determine their comparative advantage against their single-task counterparts. A total of 12 teams participated in the challenge, contributing 7 action recognition methods, 9 instrument segmentation techniques, and 4 multitask approaches that integrated both action recognition and instrument segmentation. The complete SAR-RARP50 dataset is available at: https://rdr.ucl.ac.uk/projects/SARRARP50_Segmentation_of_surgical_instrumentation_and_Action_Recognition_on_Robot-Assisted_Radical_Prostatectomy_Challenge/191091

Neutrinos in core-collapse supernovae and neutron-star mergers are susceptible to flavor instabilities of three kinds: slow, fast, and collisional. Prior work has established mappings of the first two onto abstract mechanical systems in flavor space, respectively named the slow and fast flavor pendula. Here we introduce and analyze the flavor pendulum associated with the third class. We explain our results in terms of the recently developed theory of neutrino quantum thermodynamics. Perhaps our most surprising finding is that there exists a limit in which decoherent interactions drive perfectly coherent flavor conversion., Comment: 10 pages, 4 figures

We propose Masked-Attention Transformers for Surgical Instrument Segmentation (MATIS), a two-stage, fully transformer-based method that leverages modern pixel-wise attention mechanisms for instrument segmentation. MATIS exploits the instance-level nature of the task by employing a masked attention module that generates and classifies a set of fine instrument region proposals. Our method incorporates long-term video-level information through video transformers to improve temporal consistency and enhance mask classification. We validate our approach in the two standard public benchmarks, Endovis 2017 and Endovis 2018. Our experiments demonstrate that MATIS' per-frame baseline outperforms previous state-of-the-art methods and that including our temporal consistency module boosts our model's performance further., Comment: ISBI 2023 (Oral). Winning method of the 2022 SAR-RARP50 Challenge (arXiv:2401.00496). Official extension published at arXiv:2401.11174 . Code available at https://github.com/BCV-Uniandes/MATIS

Electron-spin resonance carried out with scanning tunneling microscopes (ESR-STM) is a recently developed experimental technique that is attracting enormous interest on account of its potential to carry out single-spin on-surface resonance with subatomic resolution. Here we carry out a theoretical study of the role of tip-adatom interactions and provide guidelines for choosing the experimental parameters in order to optimize spin resonance measurements. We consider the case of the Fe adatom on a MgO surface and its interaction with the spin-polarized STM tip. We address three problems: first, how to optimize the tip-sample distance to cancel the effective magnetic field created by the tip on the surface spin, in order to carry out proper magnetic field sensing. Second, how to reduce the voltage dependence of the surface-spin resonant frequency, in order to minimize tip-induced decoherence due to voltage noise. Third, we propose an experimental protocol to infer the detuning angle between the applied field and the tip magnetization, which plays a crucial role in the modeling of the experimental results., Comment: 10 pages, 5 Figures

We address the problem of electronic and nuclear spin resonance of an individual atom on a surface driven by a scanning tunnelling microscope. Several mechanisms have been proposed so far, some of them based on the modulation of exchange and crystal field associated to a piezoelectric displacement of the adatom driven by the RF tip electric field. Here we consider a new mechanism, where the piezoelectric displacement modulates the g factor anisotropy, leading both to electronic and nuclear spin flip transitions. We discuss thoroughly the cases of Ti-H (S = 1/2) and Fe (S = 2) on MgO, relevant for recent experiments. We model the system using two approaches. First, an analytical model that includes crystal field, spin orbit coupling and hyperfine interactions. Second, we carry out density functional based calculations. We find that the modulation of the anisotropy of the g tensor due to the piezoelectric displacement of the atom is an additional mechanism for STM based single spin resonance, that would be effective in S = 1/2 adatoms with large spin orbit coupling. In the case of Ti-H on MgO, we predict a modulation spin resonance frequency driven by the DC electric field of the tip., Comment: 15 pages, 7 figures