Search

With increasing numbers of patients and novel drugs for distinct causes of systolic and diastolic heart failure, automated assessment of cardiac function is important. We aimed to provide a non-invasive method to predict diagnosis of patients undergoing cardiac MRI (cMRI) and to obtain left ventricular end-diastolic pressure (LVEDP).

Purpose: Minimally invasive mitral valve surgery (MIMVS) and transcatheter edge-to-edge repair (TEER) are complex procedures used to treat mitral valve (MV) pathologies, but with limited training opportunities available. To enable training, a realistic hemodynamic environment is needed. In this work we aimed to develop and validate a simulator that enables investigation of MV pathologies and their repair by MIMVS and TEER in a hemodynamic setting. Methods: Different MVs were installed in the simulator, and pressure, flow, and transesophageal echocardiographic measurements were obtained. To confirm the simulator’s physiological range, we first installed a biological prosthetic, a mechanical prosthetic, and a competent excised porcine MV. Subsequently, we inserted two porcine MVs—one with induced chordae tendineae rupture and the other with a dilated annulus, along with a patient-specific silicone valve extracted from echocardiography with bi-leaflet prolapse. Finally, TEER and MIMVS procedures were conducted by experts to repair the MVs. Results: Systolic pressures, cardiac outputs, and regurgitations volumes (RVol) with competent MVs were 119 ± 1 mmHg, 4.78 ± 0.16 l min−1, and 5 ± 3 ml respectively, and thus within the physiological range. In contrast, the pathological MVs displayed increased RVols. MIMVS and TEER resulted in a decrease in RVols and mitigated the severity of mitral regurgitation. Conclusion: Ex-vivo modelling of MV pathologies and repair procedures using the described simulator realistically replicated physiological in-vivo conditions. Furthermore, we showed the feasibility of performing MIMVS and TEER at the simulator, also at patient-specific level, thus providing new clinical perspectives in terms of training modalities and personalized planning.

Purpose: Complicated type B Aortic dissection is a severe aortic pathology that requires treatment through thoracic endovascular aortic repair (TEVAR). During TEVAR a stentgraft is deployed in the aortic lumen in order to restore blood flow. Due to the complicated pathology including an entry, a resulting dissection wall with potentially several re-entries, replicating this structure artificially has proven to be challenging thus far. Methods: We developed a 3d printed, patient-specific and perfused aortic dissection phantom with a flexible dissection flap and all major branching vessels. The model was segmented from CTA images and fabricated out of a flexible material to mimic aortic wall tissue. It was placed in a pulsatile hemodynamic flow loop. Hemodynamics were investigated through pressure and flow measurements and doppler ultrasound imaging. Surgeons performed a TEVAR intervention including stentgraft deployment under fluoroscopic guidance. Results: The flexible aortic dissection phantom was successfully incorporated in the hemodynamic flow loop, a systolic pressure of 112 mmHg and physiological flow of 4.05 L per minute was reached. Flow velocities were higher in true lumen with a up to 35.7 cm/s compared to the false lumen with a maximum of 13.3 cm/s, chaotic flow patterns were observed on main entry and reentry sights. A TEVAR procedure was successfully performed under fluoroscopy. The position of the stentgraft was confirmed using CTA imaging. Conclusions: This perfused in-vitro phantom allows for detailed investigation of the complex inner hemodynamics of aortic dissections on a patient-specific level and enables the simulation of TEVAR procedures in a real endovascular operating environment. Therefore, it could provide a dynamic platform for future surgical training and research.

Purpose: Machine learning approaches can only be reliably evaluated if training, validation, and test data splits are representative and not affected by the absence of classes. Surgical workflow and instrument recognition are two tasks that are complicated in this manner, because of heavy data imbalances resulting from different length of phases and their potential erratic occurrences. Furthermore, sub-properties like instrument (co-)occurrence are usually not particularly considered when defining the split. Methods: We present a publicly available data visualization tool that enables interactive exploration of dataset partitions for surgical phase and instrument recognition. The application focuses on the visualization of the occurrence of phases, phase transitions, instruments, and instrument combinations across sets. Particularly, it facilitates assessment of dataset splits, especially regarding identification of sub-optimal dataset splits. Results: We performed analysis of the datasets Cholec80, CATARACTS, CaDIS, M2CAI-workflow, and M2CAI-tool using the proposed application. We were able to uncover phase transitions, individual instruments, and combinations of surgical instruments that were not represented in one of the sets. Addressing these issues, we identify possible improvements in the splits using our tool. A user study with ten participants demonstrated that the participants were able to successfully solve a selection of data exploration tasks. Conclusion: In highly unbalanced class distributions, special care should be taken with respect to the selection of an appropriate dataset split because it can greatly influence the assessments of machine learning approaches. Our interactive tool allows for determination of better splits to improve current practices in the field. The live application is available at https://cardio-ai.github.io/endovis-ml/.

ABSTRACTMulti-point tracking is a challenging task that involves detecting points in the scene and tracking them across time. Here, metrics from Multi-object tracking (MOT) methods are shown to perform better than frame-based F-measures. The recently proposed HOTA metric, used for benchmarks such as the KITTI dataset, better evaluates the performance over metrics like MOTA, DetA, and IDF1. While HOTA takes into account temporal associations, it does not provide a tailored means to analyse the spatial associations of a dataset in a multi-camera setup. Moreover, there are differences in evaluating the detection task for points vs. objects (point distances vs. bounding box overlap). Therefore, we propose a multi-view higher-order tracking metric mvHOTA,to determine the accuracy of multi-point (multi-instance and multi-class) tracking methods while taking into account temporal and spatial associations. We demonstrate its use in evaluating the tracking performance on an endoscopic point detection dataset from a previously organised surgical data science challenge. Furthermore, we compare with other adjusted MOT metrics for this use-case, discuss the properties of mvHOTA, and show how the proposed multi-view Association and the Occlusion index (OI) facilitate analysis of methods with respect to handling of occlusions. The code is available at https://github.com/Cardio-AI/mvhota.

Purpose: Minimally invasive surgeries have restricted surgical ports, demanding a high skill level from the surgeon. Surgical simulation potentially reduces this steep learning curve and additionally provides quantitative feedback. Markerless depth sensors show great promise for quantification, but most such sensors are not designed for accurate reconstruction of complex anatomical forms in close-range. Methods: This work compares three commercially available depth sensors, namely the Intel D405, D415, and the Stereolabs Zed-Miniin the range of 12–20 cm, for use in surgical simulation. Three environments are designed that closely mimic surgical simulation, comprising planar surfaces, rigid objects, and mitral valve models of silicone and realistic porcine tissue. The cameras are evaluated on Z-accuracy, temporal noise, fill rate, checker distance, point cloud comparisons, and visual inspection of surgical scenes, across several camera settings. Results: The Intel cameras show sub-mm accuracy in most static environments. The D415 fails in reconstructing valve models, while the Zed-Mini provides lesser temporal noise and higher fill rate. The D405 could reconstruct anatomical structures like the mitral valve leaflet and a ring prosthesis, but performs poorly for reflective surfaces like surgical tools and thin structures like sutures. Conclusion: If a high temporal resolution is needed and lower spatial resolution is acceptable, the Zed-Mini is the best choice, whereas the Intel D405 is the most suited for close-range applications. The D405 shows potential for applications like deformable registration of surfaces, but is not yet suitable for applications like real-time tool tracking or surgical skill assessment.

Purpose: Integrated operating rooms provide rich sources of temporal information about surgical procedures, which has led to the emergence of surgical data science. However, little emphasis has been put on interactive visualization of such temporal datasets to gain further insights. Our goal is to put heterogeneous data sequences in relation to better understand the workflows of individual procedures as well as selected subsets, e.g., with respect to different surgical phase distributions and surgical instrument usage patterns. Methods: We developed a reusable web-based application design to analyze data derived from surgical procedure recordings. It consists of aggregated, synchronized visualizations for the original temporal data as well as for derived information, and includes tailored interaction techniques for selection and filtering. To enable reproducibility, we evaluated it across four types of surgeries from two openly available datasets (HeiCo and Cholec80). User evaluation has been conducted with twelve students and practitioners with surgical and technical background. Results: The evaluation showed that the application has the complexity of an expert tool (System Usability Score of 57.73) but allowed the participants to solve various analysis tasks correctly (78.8% on average) and to come up with novel hypotheses regarding the data. Conclusion: The novel application supports postoperative expert-driven analysis, improving the understanding of surgical workflows and the underlying datasets. It facilitates analysis across multiple synchronized views representing information from different data sources and, thereby, advances the field of surgical data science.

Purpose:: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method:: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussianlayer followed by a differentiable 2D spatial Soft-Argmaxlayer to function as a local non-maximum suppression. Results:: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean $F_1$\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}of $+0.0422$\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+ 0.0422$$\end{document}over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean $F_1$\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}of $+0.0865$\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+ 0.0865$$\end{document}over the baseline. Conclusion:: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.

Cardiovascular diseases remain a significant health burden, with imaging modalities like echocardiography, cardiac computed tomography, and cardiac magnetic resonance imaging playing a crucial role in diagnosis and prognosis. However, the inherent heterogeneity of these diseases poses challenges, necessitating advanced analytical methods like radiomics and artificial intelligence. Radiomics extracts quantitative features from medical images, capturing intricate patterns and subtle variations that may elude visual inspection. Artificial intelligence techniques, including deep learning, can analyze these features to generate knowledge, define novel imaging biomarkers, and support diagnostic decision-making and outcome prediction. Radiomics and artificial intelligence thus hold promise for significantly enhancing diagnostic and prognostic capabilities in cardiac imaging, paving the way for more personalized and effective patient care. This review explores the synergies between radiomics and artificial intelligence in cardiac imaging, following the radiomics workflow and introducing concepts from both domains. Potential clinical applications, challenges, and limitations are discussed, along with solutions to overcome them.

Purpose: Intensive planning and analysis from echocardiography are a crucial step before reconstructive surgeries are applied to malfunctioning mitral valves. Volume visualizations of echocardiographic data are often used in clinical routine. However, they lack a clear visualization of the crucial factors for decision making. Methods: We build upon patient-specific mitral valve surface models segmented from echocardiography that represent the valve’s geometry, but suffer from self-occlusions due to complex 3D shape. We transfer these to 2D maps by unfolding their geometry, resulting in a novel 2D representation that maintains anatomical resemblance to the 3D geometry. It can be visualized together with color mappings and presented to physicians to diagnose the pathology in one gaze without the need for further scene interaction. Furthermore, it facilitates the computation of a Pathology Score, which can be used for diagnosis support. Results: Quality and effectiveness of the proposed methods were evaluated through a user survey conducted with domain experts. We assessed pathology detection accuracy using 3D valve models in comparison with the novel visualizations. Classification accuracy increased by 5.3% across all tested valves and by 10.0% for prolapsed valves. Further, the participants’ understanding of the relation between 3D and 2D views was evaluated. The Pathology Scoreis found to have potential to support discriminating pathologic valves from normal valves. Conclusions: In summary, our survey shows that pathology detection can be improved in comparison with simple 3D surface visualizations of the mitral valve. The correspondence between the 2D and 3D representations is comprehensible, and color-coded pathophysiological magnitudes further support the clinical assessment.

Given the multitude of challenges surgeons face during mitral valve repair surgery, they should have a high confidence in handling of instruments and in the application of surgical techniques before they enter the operating room. Unfortunately, opportunities for surgical training of minimally invasive repair are very limited, leading to a situation where most surgeons undergo a steep learning curve while operating the first patients. In order to provide a realistic tool for surgical training, a commercial simulator was augmented by flexible patient-specific mitral valve replica. In an elaborated production pipeline, finalized after many optimization cycles, models were segmented from 3D ultrasound and then 3D-printable molds were computed automatically and printed in rigid material, the lower part being water-soluble. After silicone injection, the silicone model was dissolved from the mold and anchored in the simulator. To our knowledge, our models are the first to comprise the full mitral valve apparatus, i.e., the annulus, leaflets, chordae tendineae and papillary muscles. Nine different valve molds were automatically created according to the proposed workflow (seven prolapsed valves and two valves with functional mitral insufficiency). From these mold geometries, 16 replica were manufactured. A material test revealed that EcoflexTM00-30is the most suitable material for leaflet-mimicking tissue out of seven mixtures. Production time was around 36 h per valve. Twelve surgeons performed various surgical techniques, e.g., annuloplasty, neo-chordae implantation, triangular leaflet resection, and assessed the realism of the valves very positively. The standardized production process guarantees a high anatomical recapitulation of the silicone valves to the segmented models and the ultrasound data. Models are of unprecedented quality and maintain a high realism during haptic interaction with instruments and suture material.

Augmented reality (AR) has emerged as a promising approach to support surgeries; however, its application in real world scenarios is still very limited. Besides sophisticated registration tasks that need to be solved, surgical AR visualizations have not been studied in a standardized and comparative manner. To foster the development of future AR applications, a steerable framework is urgently needed to rapidly evaluate new visualization techniques, explore their individual parameter spaces and define relevant application scenarios. Inspired by its beneficial usage in the automotive industry, the underlying concept of virtual reality (VR) is capable of transforming complex real environments into controllable virtual ones. We present an interactive VR framework, called Augmented Visualization Box (AVB), in which visualizations for AR can be systematically investigated without explicitly performing an error-prone registration. As use case, a virtual laparoscopic scenario with anatomical surface models was created in a computer game engine. In a study with eleven surgeons, we analyzed this VR setting under different environmental factors and its applicability for a quantitative assessment of different AR overlay concepts. According to the surgeons, the visual impression of the VR scene is mostly influenced by 2D surface details and lighting conditions. The AR evaluation shows that, depending on the visualization used and its capability to encode depth, 37% to 91% of the experts made wrong decisions, but were convinced of their correctness. These results show that surgeons have more confidence in their decisions, although they are wrong, when supported by AR visualizations. With AVB, intraoperative situations are realistically simulated to quantitatively benchmark current AR overlay methods. Successful surgical task execution in an AR system can only be facilitated if visualizations are customized toward the surgical task.

Mitral valve reconstruction is a widespread surgical method to repair incompetent mitral valves, which usually includes implantation of a ring prosthesis. To date, intraoperative analysis of the mitral valve is merely based on visual assessment using simple surgical tools, which might not allow for accurate assessment of the complex anatomy. We propose a novel intraoperative computer-based assistance system, which combines passive optical tracking technology with tailored measurement strategies applicable during different phases of the intraoperative workflow. Based on the assessment of the valvular apparatus by customized tracked instruments, the system (1) generates an enhanced three-dimensional visualization, which (2) incorporates accurate quantifications and (3) provides assistance, e.g., in terms of virtual prosthesis selection. Phantom experiments in a realistic environment revealed a high system accuracy (mean precision $$0.12 \pm 0.09$$ 0.12±0.09 mm and mean trueness $$0.77 \pm 0.39$$ 0.77±0.39 mm) and a low user error (mean precision $$0.18 \pm 0.10 $$ 0.18±0.10 mm and mean trueness $$0.81 \pm 0.36$$ 0.81±0.36 mm). The assistance system was successfully applied five times during open and minimally invasive reconstructive surgery in patients having mitral valve insufficiency. The measurement steps integrate well into the traditional workflow, enhancing the surgeon’s three-dimensional perception and generating a suggestion for an appropriate prosthesis. The proposed assistance system provides a novel, accurate, and reproducible method for assessing the valvular geometry intraoperatively.

Introduction:“Le Coeur en Sabot” (or the boot-shaped heart) is a radiological description of the appearance of the heart on a plain radiograph of a patient with Tetralogy of Fallot. This highlights the importance of the shape of the heart. Maladaptive remodeling is known to occur in patients with repaired Tetralogy of Fallot (rToF) due to residual lesions such as Pulmonary Regurgitant Fraction (PRF)Hypothesis:Shape and function analysis can reveal novel remodeling patterns associated with adverse events in patients with rToFMethods:Biventricular shape and function were studied in 192 patients with rToF (median age 15 years). Linear discriminative analysis (LDA) and principal component analysis (PCA) were used to identify shape differences between patients with and without adverse events (AE). AE included death, arrhythmias, and cardiac arrest (median follow-up 10 years)Results:LDA and PCA showed that shape characteristics pertaining to adverse events included a more circular LV (decreased eccentricity), dilated (increased sphericity) LV base, increased RV apical sphericity, and decreased RV basal sphericity. Multivariate LDA showed that the optimal discriminative model included only RV apical ejection fraction and one PCA mode associated with a more circular and dilated LV base (AUC = 0.78). PRF did not add value, and shape changes associated with increased PRF were not predictive of AE.Conclusions:Remodeling patterns in patients with rToF are associated with AE, independent of PRF. Mechanisms leading to AE include LV basal dilation with a reduced RV apical ejection fraction. That is to say that the old descriptor, “Le Coeur en Sabot'' may also describe that shape that is adverse in rTOF. The toe of this boot would be the dilated and poorly contractile RV apex and the ankle would be the spherical LV base (Figure 1: Top Row Diastole. Middle Systole. Posterior projection (a), anatomical position (b) and superior (c). LV is green)