Search

Surgical scene perception via videos is critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets face challenges such as small scale, lack of diversity in surgery and phase categories, and absence of time-localized annotations. These limitations impede action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 fine-grained operations. 2) Sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability. 3) Time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 285 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Code and dataset are available at: https://minghu0830.github.io/OphNet-benchmark/., Comment: Accepted by ECCV 2024

Diffusion probabilistic models (DPMs) which employ explicit likelihood characterization and a gradual sampling process to synthesize data, have gained increasing research interest. Despite their huge computational burdens due to the large number of steps involved during sampling, DPMs are widely appreciated in various medical imaging tasks for their high-quality and diversity of generation. Magnetic resonance imaging (MRI) is an important medical imaging modality with excellent soft tissue contrast and superb spatial resolution, which possesses unique opportunities for DPMs. Although there is a recent surge of studies exploring DPMs in MRI, a survey paper of DPMs specifically designed for MRI applications is still lacking. This review article aims to help researchers in the MRI community to grasp the advances of DPMs in different applications. We first introduce the theory of two dominant kinds of DPMs, categorized according to whether the diffusion time step is discrete or continuous, and then provide a comprehensive review of emerging DPMs in MRI, including reconstruction, image generation, image translation, segmentation, anomaly detection, and further research topics. Finally, we discuss the general limitations as well as limitations specific to the MRI tasks of DPMs and point out potential areas that are worth further exploration.

Estimating parameters from data is a fundamental problem in physics, customarily done by minimizing a loss function between a model and observed statistics. In scattering-based analysis, researchers often employ their domain expertise to select a specific range of wavevectors for analysis, a choice that can vary depending on the specific case. We introduce another paradigm that defines a probabilistic generative model from the beginning of data processing and propagates the uncertainty for parameter estimation, termed ab initio uncertainty quantification (AIUQ). As an illustrative example, we demonstrate this approach with differential dynamic microscopy (DDM) that extracts dynamical information through Fourier analysis at a selected range of wavevectors. We first show that DDM is equivalent to fitting a temporal variogram in the reciprocal space using a latent factor model as the generative model. Then we derive the maximum marginal likelihood estimator, which optimally weighs information at all wavevectors, therefore eliminating the need to select the range of wavevectors. Furthermore, we substantially reduce the computational cost by utilizing the generalized Schur algorithm for Toeplitz covariances without approximation. Simulated studies validate that AIUQ significantly improves estimation accuracy and enables model selection with automated analysis. The utility of AIUQ is also demonstrated by three distinct sets of experiments: first in an isotropic Newtonian fluid, pushing limits of optically dense systems compared to multiple particle tracking; next in a system undergoing a sol-gel transition, automating the determination of gelling points and critical exponent; and lastly, in discerning anisotropic diffusive behavior of colloids in a liquid crystal. These outcomes collectively underscore AIUQ's versatility to capture system dynamics in an efficient and automated manner., Comment: 23 pages, 9 figures

The dynamics of cellular pattern formation is crucial for understanding embryonic development and tissue morphogenesis. Recent studies have shown that human dermal fibroblasts cultured on liquid crystal elastomers can exhibit an increase in orientational alignment over time, accompanied by cell proliferation, under the influence of the weak guidance of a molecularly aligned substrate. However, a comprehensive understanding of how this order arises remains largely unknown. This knowledge gap may be attributed, in part, to a scarcity of mechanistic models that can capture the temporal progression of the complex nonequilibrium dynamics during the cellular alignment process. The orientational alignment occurs primarily when cells reach a high density near confluence. Therefore, for accurate modeling, it is crucial to take into account both the cell-cell interaction term and the influence from the substrate, acting as a one-body external potential term. To fill in this gap, we develop a hybrid procedure that utilizes statistical learning approaches to extend the state-of-the-art physics models for quantifying both effects. We develop a more efficient way to perform feature selection that avoids testing all feature combinations through simulation. The maximum likelihood estimator of the model was derived and implemented in computationally scalable algorithms for model calibration and simulation. By including these features, such as the non-Gaussian, anisotropic fluctuations, and limiting alignment interaction only to neighboring cells with the same velocity direction, this model quantitatively reproduce the key system-level parameters--the temporal progression of the velocity orientational order parameters and the variability of velocity vectors, whereas models missing any of the features fail to capture these temporally dependent parameters., Comment: 20 pages, 14 figures

The ability of cells to reorganize in response to external stimuli is important in areas ranging from morphogenesis to tissue engineering. Elongated cells can co-align due to steric effects, forming states with local order. We show that molecular-scale substrate anisotropy can direct cell organization, resulting in the emergence of nematic order on tissue scales. To quantitatively examine the disorder-order transition, we developed a high-throughput imaging platform to analyze velocity and orientational correlations for several thousand cells over days. The establishment of global, seemingly long-ranged order is facilitated by enhanced cell division along the substrate's nematic axis, and associated extensile stresses that restructure the cells' actomyosin networks. Our work, which connects to a class of systems known as active dry nematics, provides a new understanding of the dynamics of cellular remodeling and organization in weakly interacting cell collectives. This enables data-driven discovery of cell-cell interactions and points to strategies for tissue engineering., Comment: 29 pages, 7 figures

Physically intelligent micro-robotic systems exploit information embedded in micro-robots, their colloidal cargo, and their milieu to interact, assemble and form functional structures. Nonlinear anisotropic fluids like nematic liquid crystals (NLCs) provide untapped opportunities to embed interactions via their topological defects, complex elastic responses, and their ability to dramatically restructure in dynamic settings. Here we design and fabricate a 4-armed ferromagnetic micro-robot to embed and dynamically reconfigure information in the nematic director field, generating a suite of physical interactions for cargo manipulation. The micro-robot shape and surface chemistry are designed to generate a nemato-elastic energy landscape in the domain that defines multiple modes of emergent, bottom-up interactions with passive colloids. Micro-robot rotation expands the ability to sculpt interactions; the energy landscape around a rotating micro-robot is dynamically reconfigured by complex far-from-equilibrium dynamics of the micro-robot's companion topological defect. These defect dynamics allow transient information to be programmed into the domain and exploited for top-down cargo manipulation. We demonstrate robust micro-robotic manipulation strategies that exploit these diverse modes of nemato-elastic interaction to achieve cargo docking, transport, release, and assembly of complex reconfigurable structures at multi-stable sites. Such structures are of great interest to future developments of LC-based advanced optical device and micro-manufacturing in anisotropic environments., Comment: 23 pages, 8 figures in the main text and 4 figures in the SI

Non-equilibrium dynamics of topological defects can be used as a fundamental propulsion mechanism in microscopic active matter. Here, we demonstrate swimming of topological defect-propelled colloidal particles in (passive) nematic fluids through experiments and numerical simulations. Dynamic swim strokes of the topological defects are driven by colloidal rotation in an external magnetic field, causing periodic defect rearrangement which propels the particles. The swimming velocity is determined by the colloid's angular velocity, sense of rotation and defect polarity. By controlling them we can locomote the particles along different trajectories. We demonstrate scattering -- that is, the effective pair interactions -- of two of our defect-propelled swimmers, which we show is highly anisotropic and depends on the microscopic structure of the defect stroke, including the local defect topology and polarity. More generally, this work aims to develop biomimetic active matter based on the underlying relevance of topology.