Search

• A novel loss function for multi-organ segmentation. • Handling both input and output class imbalance. • Smoothing dice (or similar discrete) loss function(s). • Preventing potential gradient vanishing/exploding problem caused by Dice or similar loss functions. • First deep network for PET multi-organ segmentation. • Tested on various data-sets: MRI, PET, and CT. Simultaneous segmentation of multiple organs from different medical imaging modalities is a crucial task as it can be utilized for computer-aided diagnosis, computer-assisted surgery, and therapy planning. Thanks to the recent advances in deep learning, several deep neural networks for medical image segmentation have been introduced successfully for this purpose. In this paper, we focus on learning a deep multi-organ segmentation network that labels voxels. In particular, we examine the critical choice of a loss function in order to handle the notorious imbalance problem that plagues both the input and output of a learning model. The input imbalance refers to the class-imbalance in the input training samples (i.e., small foreground objects embedded in an abundance of background voxels, as well as organs of varying sizes). The output imbalance refers to the imbalance between the false positives and false negatives of the inference model. In order to tackle both types of imbalance during training and inference, we introduce a new curriculum learning based loss function. Specifically, we leverage Dice similarity coefficient to deter model parameters from being held at bad local minima and at the same time gradually learn better model parameters by penalizing for false positives/negatives using a cross entropy term. We evaluated the proposed loss function on three datasets: whole body positron emission tomography (PET) scans with 5 target organs, magnetic resonance imaging (MRI) prostate scans, and ultrasound echocardigraphy images with a single target organ i.e., left ventricular. We show that a simple network architecture with the proposed integrative loss function can outperform state-of-the-art methods and results of the competing methods can be improved when our proposed loss is used. [ABSTRACT FROM AUTHOR]

As a minimally invasive surgery to treat atrial fibrillation (AF), catheter based ablation uses high radio-frequency energy to eliminate potential sources of abnormal electrical events, especially around the ostia of pulmonary veins (PV). Fusing a patient-specific left atrium (LA) model (including LA chamber, appendage, and PVs) with electro-anatomical maps or overlaying the model onto 2-D real-time fluoroscopic images provides valuable visual guidance during the intervention. In this work, we present a fully automatic LA segmentation system on nongated C-arm computed tomography (C-arm CT) data, where thin boundaries between the LA and surrounding tissues are often blurred due to the cardiac motion artifacts. To avoid segmentation leakage, the shape prior should be exploited to guide the segmentation. A single holistic shape model is often not accurate enough to represent the whole LA shape population under anatomical variations, e.g., the left common PVs vs. separate left PVs. Instead, a part based LA model is proposed, which includes the chamber, appendage, four major PVs, and right middle PVs. Each part is a much simpler anatomical structure compared to the holistic one and can be segmented using a model-based approach (except the right middle PVs). After segmenting the LA parts, the gaps and overlaps among the parts are resolved and segmentation of the ostia region is further refined. As a common anatomical variation, some patients may contain extra right middle PVs, which are segmented using a graph cuts algorithm under the constraints from the already extracted major right PVs. Our approach is computationally efficient, taking about 2.6 s to process a volume with 256\,\times\,256\,\times\,245 voxels. Experiments on 687 C-arm CT datasets demonstrate its robustness and state-of-the-art segmentation accuracy. [ABSTRACT FROM AUTHOR]

Objectives: The molecular subtyping of diffuse gliomas is important. The aim of this study was to establish predictive models based on preoperative multiparametric MRI. Methods: A total of 1016 diffuse glioma patients were retrospectively collected from Beijing Tiantan Hospital. Patients were randomly divided into the training (n = 780) and validation (n = 236) sets. According to the 2016 WHO classification, diffuse gliomas can be classified into four binary classification tasks (tasks I–IV). Predictive models based on radiomics and deep convolutional neural network (DCNN) were developed respectively, and their performances were compared with receiver operating characteristic (ROC) curves. Additionally, the radiomics and DCNN features were visualized and compared with the t-distributed stochastic neighbor embedding technique and Spearman's correlation test. Results: In the training set, areas under the curves (AUCs) of the DCNN models (ranging from 0.99 to 1.00) outperformed the radiomics models in all tasks, and the accuracies of the DCNN models (ranging from 0.90 to 0.94) outperformed the radiomics models in tasks I, II, and III. In the independent validation set, the accuracies of the DCNN models outperformed the radiomics models in all tasks (0.74–0.83), and the AUCs of the DCNN models (0.85–0.89) outperformed the radiomics models in tasks I, II, and III. DCNN features demonstrated more superior discriminative capability than the radiomics features in feature visualization analysis, and their general correlations were weak. Conclusions: Both the radiomics and DCNN models could preoperatively predict the molecular subtypes of diffuse gliomas, and the latter performed better in most circumstances. Key Points: • The molecular subtypes of diffuse gliomas could be predicted with MRI. • Deep learning features tend to outperform radiomics features in large cohorts. • The correlation between the radiomics features and DCNN features was low. [ABSTRACT FROM AUTHOR]

Today, quantitative analysis of three-dimensional (3D) dynamics of the left ventricle (LV) cannot be performed directly in the catheter lab using a current angiographic C-arm system, which is the workhorse imaging modality for cardiac interventions. Therefore, myocardial wall analysis is completely based on the 2D angiographic images or pre-interventional 3D/4D imaging. In this paper, we present a complete framework to study the ventricular wall motion in 4D (3D+t) directly in the catheter lab. From the acquired 2D projection images, a dynamic 3D surface model of the LV is generated, which is then used to detect ventricular dyssynchrony. Different quantitative features to evaluate LV dynamics known from other modalities (ultrasound, magnetic resonance imaging) are transferred to the C-arm CT data. We use the ejection fraction, the systolic dyssynchrony index a 3D fractional shortening and the phase to maximal contraction (ϕi, max) to determine an indicator of LV dyssynchrony and to discriminate regionally pathological from normal myocardium. The proposed analysis tool was evaluated on simulated phantom LV data with and without pathological wall dysfunctions. The LV data used is publicly available online at https://conrad.stanford.edu/data/heart. In addition, the presented framework was tested on eight clinical patient data sets. The first clinical results demonstrate promising performance of the proposed analysis tool and encourage the application of the presented framework to a larger study in clinical practice. [ABSTRACT FROM AUTHOR]

Segmenting prostate from magnetic resonance imaging (MRI) is a critical procedure in prostate cancer staging and treatment planning. Considering the nature of labeled data scarcity for medical images, semi-supervised learning (SSL) becomes an appealing solution since it can simultaneously exploit limited labeled data and a large amount of unlabeled data. However, SSL relies on the assumption that the unlabeled images are abundant, which may not be satisfied when the local institute has limited image collection capabilities. An intuitive solution is to seek support from other centers to enrich the unlabeled image pool. However, this further introduces data heterogeneity, which can impede SSL that works under identical data distribution with certain model assumptions. Aiming at this under-explored yet valuable scenario, in this work, we propose a separated collaborative learning (SCL) framework for semi-supervised prostate segmentation with multi-site unlabeled MRI data. Specifically, on top of the teacher-student framework, SCL exploits multi-site unlabeled data by: (i) Local learning, which advocates local distribution fitting, including the pseudo label learning that reinforces confirmation of low-entropy easy regions and the cyclic propagated real label learning that leverages class prototypes to regularize the distribution of intra-class features; (ii) External multi-site learning, which aims to robustly mine informative clues from external data, mainly including the local-support category mutual dependence learning, which takes the spirit that mutual information can effectively measure the amount of information shared by two variables even from different domains, and the stability learning under strong adversarial perturbations to enhance robustness to heterogeneity. Extensive experiments on prostate MRI data from six different clinical centers show that our method can effectively generalize SSL on multi-site unlabeled data and significantly outperform other semi-supervised segmentation methods. Besides, we validate the extensibility of our method on the multi-class cardiac MRI segmentation task with data from four different clinical centers. • We present a new and practical scenario of multi-site semi-supervised learning (MS-SSL), which allows the enrichment of the unlabeled pool with heterogeneous unlabeled data from multiple arbitrary sites and support the semi-supervised learning in local centers. • We propose a new separated collaborative learning (SCL) framework, including local learning and external multi-site learning, for this under-explored scenario. • We propose a novel local-support category mutual dependence learning scheme, which advocates mutual information-based distribution-insensitive relationship modeling on region-of-interests, for effective collaboration between local labeled data and heterogeneous external unlabeled data to support local learning. • Our method is extensively evaluated on public prostate MRI datasets from six different institutes with varying scanning protocols and patient demographics. The experimental results demonstrate the superiority of our approach. • We also validated the extensibility of our method on the multi-class cardiac MRI segmentation task with data from four different clinical centers. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation (UDA) methods have shown their promising performance in the cross-modality medical image segmentation tasks. These typical methods usually utilize a translation network to transform images from the source domain to target domain or train the pixel-level classifier merely using translated source images and original target images. However, when there exists a large domain shift between source and target domains, we argue that this asymmetric structure, to some extent, could not fully eliminate the domain gap. In this paper, we present a novel deep symmetric architecture of UDA for medical image segmentation, which consists of a segmentation sub-network, and two symmetric source and target domain translation sub-networks. To be specific, based on two translation sub-networks, we introduce a bidirectional alignment scheme via a shared encoder and two private decoders to simultaneously align features 1) from source to target domain and 2) from target to source domain, which is able to effectively mitigate the discrepancy between domains. Furthermore, for the segmentation sub-network, we train a pixel-level classifier using not only original target images and translated source images, but also original source images and translated target images, which could sufficiently leverage the semantic information from the images with different styles. Extensive experiments demonstrate that our method has remarkable advantages compared to the state-of-the-art methods in three segmentation tasks, such as cross-modality cardiac, BraTS, and abdominal multi-organ segmentation. [ABSTRACT FROM AUTHOR]

Medical image segmentation has achieved remarkable advancements using deep neural networks (DNNs). However, DNNs often need big amounts of data and annotations for training, both of which can be difficult and costly to obtain. In this work, we propose a unified framework for generalized low-shot (one- and few-shot) medical image segmentation based on distance metric learning (DML). Unlike most existing methods which only deal with the lack of annotations while assuming abundance of data, our framework works with extreme scarcity of both, which is ideal for rare diseases. Via DML, the framework learns a multimodal mixture representation for each category, and performs dense predictions based on cosine distances between the pixels’ deep embeddings and the category representations. The multimodal representations effectively utilize the inter-subject similarities and intraclass variations to overcome overfitting due to extremely limited data. In addition, we propose adaptive mixing coefficients for the multimodal mixture distributions to adaptively emphasize the modes better suited to the current input. The representations are implicitly embedded as weights of the fc layer, such that the cosine distances can be computed efficiently via forward propagation. In our experiments on brain MRI and abdominal CT datasets, the proposed framework achieves superior performances for low-shot segmentation towards standard DNN-based (3D U-Net) and classical registration-based (ANTs) methods, e.g., achieving mean Dice coefficients of 81%/69% for brain tissue/abdominal multi-organ segmentation using a single training sample, as compared to 52%/31% and 72%/35% by the U-Net and ANTs, respectively. [ABSTRACT FROM AUTHOR]

Highlights • This paper shows extensive experiments to validate our method, including CT and MRI translation for pancreas segmentation and domain adaptation for mammogram X-rays for breast lesion segmentation. • This paper validates our method using a variety of advanced segmentation networks and show that our method performs generally well and consistently boosts medical 2D/3D image segmentation performance. • This paper systemically analyzes the effect of synthetic data on segmentation, with the goal to investigate the limitation of synthetic data, and inspire new scopes. Graphical abstract Abstract Synthesized medical images have several important applications. For instance, they can be used as an intermedium in cross-modality image registration or used as augmented training samples to boost the generalization capability of a classifier. In this work, we propose a generic cross-modality synthesis approach with the following targets: 1) synthesizing realistic looking 2D/3D images without needing paired training data, 2) ensuring consistent anatomical structures, which could be changed by geometric distortion in cross-modality synthesis and 3) more importantly, improving volume segmentation by using synthetic data for modalities with limited training samples. We show that these goals can be achieved with an end-to-end 2D/3D convolutional neural network (CNN) composed of mutually-beneficial generators and segmentors for image synthesis and segmentation tasks. The generators are trained with an adversarial loss, a cycle-consistency loss, and also a shape-consistency loss (supervised by segmentors) to reduce the geometric distortion. From the segmentation view, the segmentors are boosted by synthetic data from generators in an online manner. Generators and segmentors prompt each other alternatively in an end-to-end training fashion. We validate our proposed method on three datasets, including cardiovascular CT and magnetic resonance imaging (MRI), abdominal CT and MRI, and mammography X-rays from different data domains, showing both tasks are beneficial to each other and coupling these two tasks results in better performance than solving them exclusively. [ABSTRACT FROM AUTHOR]

Knowledge of left atrial (LA) anatomy is important for atrial fibrillation ablation guidance, fibrosis quantification and biophysical modelling. Segmentation of the LA from Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images is a complex problem. This manuscript presents a benchmark to evaluate algorithms that address LA segmentation. The datasets, ground truth and evaluation code have been made publicly available through the http://www.cardiacatlas.org website. This manuscript also reports the results of the Left Atrial Segmentation Challenge (LASC) carried out at the STACOM’13 workshop, in conjunction with MICCAI’13. Thirty CT and 30 MRI datasets were provided to participants for segmentation. Each participant segmented the LA including a short part of the LA appendage trunk and proximal sections of the pulmonary veins (PVs). We present results for nine algorithms for CT and eight algorithms for MRI. Results showed that methodologies combining statistical models with region growing approaches were the most appropriate to handle the proposed task. The ground truth and automatic segmentations were standardised to reduce the influence of inconsistently defined regions (e.g., mitral plane, PVs end points, LA appendage). This standardisation framework, which is a contribution of this work, can be used to label and further analyse anatomical regions of the LA. By performing the standardisation directly on the left atrial surface, we can process multiple input data, including meshes exported from different electroanatomical mapping systems. [ABSTRACT FROM AUTHOR]

Purpose: Coarctation of the aorta (CoA) is a congenital heart disease characterized by an abnormal narrowing of the proximal descending aorta. Severity of this pathology is quantified by the blood pressure drop (ΔP) across the stenotic coarctation lesion. In order to evaluate the physiological significance of the preoperative coarctation and to assess the postoperative results, the hemodynamic analysis is routinely performed by measuring the ΔP across the coarctation site via invasive cardiac catheterization. The focus of this work is to present an alternative, noninvasive measurement of blood pressure drop ΔP through the introduction of a fast, image-based workflow for personalized computational modeling of the CoA hemodynamics. Methods: The authors propose an end-to-end system comprised of shape and computational models, their personalization setup using MR imaging, and a fast, noninvasive method based on computational fluid dynamics (CFD) to estimate the pre- and postoperative hemodynamics for coarctation patients. A virtual treatment method is investigated to assess the predictive power of our approach. Results: Automatic thoracic aorta segmentation was applied on a population of 212 3D MR volumes, with mean symmetric point-to-mesh error of 3.00± 1.58 mm and average computation time of 8 s. Through quantitative evaluation of 6 CoA patients, good agreement between computed blood pressure drop and catheter measurements is shown: average differences are 2.38 ± 0.82 mm Hg (pre-), 1.10± 0.63 mm Hg (postoperative), and 4.99 ± 3.00 mm Hg (virtual stenting), respectively. Conclusions: The complete workflow is realized in a fast, mostly-automated system that is integrable in the clinical setting. To the best of our knowledge, this is the first time that three different settings (preoperative--severity assessment, poststenting--follow-up, and virtual stenting-- treatment outcome prediction) of CoA are investigated on multiple subjects. We believe that in future--given wider clinical validation--our noninvasive in-silico method could replace invasive pressure catheterization for CoA. [ABSTRACT FROM AUTHOR]

Examinations of the spinal column with both, Magnetic Resonance (MR) imaging and Computed Tomography (CT), often require a precise three-dimensional positioning, angulation and labeling of the spinal disks and the vertebrae. A fully automatic and robust approach is a prerequisite for an automated scan alignment as well as for the segmentation and analysis of spinal disks and vertebral bodies in Computer Aided Diagnosis (CAD) applications. In this article, we present a novel method that combines Marginal Space Learning (MSL), a recently introduced concept for efficient discriminative object detection, with a generative anatomical network that incorporates relative pose information for the detection of multiple objects. It is used to simultaneously detect and label the spinal disks. While a novel iterative version of MSL is used to quickly generate candidate detections comprising position, orientation, and scale of the disks with high sensitivity, the anatomical network selects the most likely candidates using a learned prior on the individual nine dimensional transformation spaces. Finally, we propose an optional case-adaptive segmentation approach that allows to segment the spinal disks and vertebrae in MR and CT respectively. Since the proposed approaches are learning-based, they can be trained for MR or CT alike. Experimental results based on 42 MR and 30 CT volumes show that our system not only achieves superior accuracy but also is among the fastest systems of its kind in the literature. On the MR data set the spinal disks of a whole spine are detected in 11.5s on average with 98.6% sensitivity and 0.073 false positive detections per volume. On the CT data a comparable sensitivity of 98.0% with 0.267 false positives is achieved. Detected disks are localized with an average position error of 2.4mm/3.2mm and angular error of 3.9°/4.5° in MR/CT, which is close to the employed hypothesis resolution of 2.1mm and 3.3°. [ABSTRACT FROM AUTHOR]