Search

We present DeepVesselNet, an architecture tailored to the challenges faced when extracting vessel networks or trees and corresponding features in 3-D angiographic volumes using deep learning. We discuss the problems of low execution speed and high memory requirements associated with full 3-D convolutional networks, high-class imbalance arising from the low percentage of vessel voxels, and unavailability of accurately annotated training data - and offer solutions as the building blocks of DeepVesselNet. First, we formulate 2-D orthogonal cross-hair filters which make use of 3-D context information at a reduced computational burden. Second, we introduce a class balancing cross-entropy loss function with false positive rate correction to handle the high-class imbalance and high false positive rate problems associated with existing loss functions. Finally, we generate synthetic dataset using a computational angiogenesis model capable of generating vascular trees under physiological constraints on local network structure and topology and use these data for transfer learning. DeepVesselNet is optimized for segmenting and analyzing vessels, and we test the performance on a range of angiographic volumes including clinical MRA data of the human brain, as well as X-ray tomographic microscopy scans of the rat brain. Our experiments show that, by replacing 3-D filters with cross-hair filters in our network, we achieve over 23% improvement in speed, lower memory footprint, lower network complexity which prevents overfitting and comparable accuracy (with a Cox-Wilcoxon paired sample significance test p-value of 0.07 when compared to full 3-D filters). Our class balancing metric is crucial for training the network and transfer learning with synthetic data is an efficient, robust, and very generalizable approach leading to a network that excels in a variety of angiography segmentation tasks., 13 pages

// Markus Wennmann 1, * , Laurent Kintzele 1, * , Marie Piraud 2 , Bjoern H. Menze 2 , Thomas Hielscher 3 , Johannes Hofmanninger 4 , Barbara Wagner 5 , Hans-Ulrich Kauczor 1 , Maximilian Merz 5 , Jens Hillengass 6 , Georg Langs 4 and Marc-Andre Weber 7 1 Diagnostic and Interventional Radiology, University Hospital Heidelberg, Heidelberg, Germany 2 Department of Computer Science, Technical University of Munich, Munich, Germany 3 Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany 4 Department of Biomedical Imaging and Image-Guided Therapy, Computational Imaging Research Laboratory, Medical University of Vienna, Vienna, Austria 5 Department of Medicine V, Multiple Myeloma Section, University of Heidelberg, Heidelberg, Germany 6 Department of Medicine, Roswell Park Comprehensive Cancer Center, Buffalo, NY, USA 7 Institute of Diagnostic and Interventional Radiology, University Medical Center Rostock, Rostock, Germany * These authors have contributed equally to this work Correspondence to: Markus Wennmann, email: Markus.Wennmann@med.uni-heidelberg.de Keywords: volumetry; speed of growth; biomarker; risk stratification; smoldering multiple myeloma Received: March 07, 2018 Accepted: April 25, 2018 Published: May 18, 2018 ABSTRACT The purpose of this study was to improve risk stratification of smoldering multiple myeloma patients, introducing new 3D-volumetry based imaging biomarkers derived from whole-body MRI. Two-hundred twenty whole-body MRIs from 63 patients with smoldering multiple myeloma were retrospectively analyzed and all focal lesions >5mm were manually segmented for volume quantification. The imaging biomarkers total tumor volume, speed of growth (development of the total tumor volume over time), number of focal lesions, development of the number of focal lesions over time and the recent imaging biomarker ‘>1 focal lesion’ of the International Myeloma Working Group were compared, taking 2-year progression rate, sensitivity and false positive rate into account. Speed of growth, using a cutoff of 114mm 3 /month, was able to isolate a high-risk group with a 2-year progression rate of 82.5%. Additionally, it showed by far the highest sensitivity in this study and in comparison to other biomarkers in the literature, detecting 63.2% of patients who progress within 2 years. Furthermore, its false positive rate (8.7%) was much lower compared to the recent imaging biomarker ‘>1 focal lesion’ of the International Myeloma Working Group. Therefore, speed of growth is the preferable imaging biomarker for risk stratification of smoldering multiple myeloma patients.

Tissue clearing methods enable the imaging of biological specimens without sectioning. However, reliable and scalable analysis of large imaging datasets in three dimensions remains a challenge. Here we developed a deep learning-based framework to quantify and analyze brain vasculature, named Vessel Segmentation & Analysis Pipeline (VesSAP). Our pipeline uses a convolutional neural network (CNN) with a transfer learning approach for segmentation and achieves human-level accuracy. By using VesSAP, we analyzed the vascular features of whole C57BL/6J, CD1 and BALB/c mouse brains at the micrometer scale after registering them to the Allen mouse brain atlas. We report evidence of secondary intracranial collateral vascularization in CD1 mice and find reduced vascularization of the brainstem in comparison to the cerebrum. Thus, VesSAP enables unbiased and scalable quantifications of the angioarchitecture of cleared mouse brains and yields biological insights into the vascular function of the brain.

Training convolutional neural networks with image-based labels leads to black-box image classification results. Saliency maps offer localization cues of class-relevant patterns, without requiring costly pixel-based labels. We show a failure mode for recently proposed weakly supervised localization models, e.g., models highlight the wrong input region, but classify correctly across all samples. Subsequently, we tested multiple architecture modifications, and propose two simple, but effective training approaches based on two-stage-learning and optional bounding box guidance, that avoid such misleading projections. Our saliency maps localize pneumonia patterns reliably and significantly better than gradCAM in terms of localization scores and expert radiologist’s ratings.

The advent of medical imaging and automatic image analysis is bringing the full quantitative assessment of lesions and tumor burden at every clinical examination within reach. This opens avenues for the development and testing of functional disease models, as well as their use in the clinical practice for personalized medicine. In this paper, we introduce a Bayesian statistical framework, based on mixed-effects models, to quantitatively test and learn functional disease models at different scales, on population longitudinal data. We also derive an effective mathematical model for the crossover between initially detected lesions and tumor dissemination, based on the Iwata-Kawasaki-Shigesada model. We finally propose to leverage this descriptive disease progression model into model-aware biomarkers for personalized risk-assessment, taking all available examinations and relevant covariates into account. As a use case, we study Multiple Myeloma, a disseminated plasma cell cancer, in which proper diagnostics is essential, to differentiate frequent precursor state without end-organ damage from the rapidly developing disease requiring therapy. After learning the best biological models for local lesion growth and global tumor burden evolution on clinical data, and computing corresponding population priors, we use individual model parameters as biomarkers, and can study them systematically for correlation with external covariates, such as sex or location of the lesion. On our cohort of 63 patients with smoldering Multiple Myeloma, we show that they perform substantially better than other radiological criteria, to predict progression into symptomatic Multiple Myeloma. Our study paves the way for modeling disease progression patterns for Multiple Myeloma, but also for other metastatic and disseminated tumor growth processes, and for analyzing large longitudinal image data sets acquired in oncological imaging. It shows the unprecedented potential of model-based biomarkers for better and more personalized treatment decisions and deserves being validated on larger cohorts to establish its role in clinical decision making.

SUMMARYTissue clearing methods enable imaging of intact biological specimens without sectioning. However, reliable and scalable analysis of such large imaging data in 3D remains a challenge. Towards this goal, we developed a deep learning-based framework to quantify and analyze the brain vasculature, named Vessel Segmentation & Analysis Pipeline (VesSAP). Our pipeline uses a fully convolutional network with a transfer learning approach for segmentation. We systematically analyzed vascular features of the whole brains including their length, bifurcation points and radius at the micrometer scale by registering them to the Allen mouse brain atlas. We reported the first evidence of secondary intracranial collateral vascularization in CD1-Elite mice and found reduced vascularization in the brainstem as compared to the cerebrum. VesSAP thus enables unbiased and scalable quantifications for the angioarchitecture of the cleared intact mouse brain and yields new biological insights related to the vascular brain function.GRAPHICAL ABSTRACTSupporting material of VesSAP is available athttp://DISCOtechnologies.org/VesSAP

Accurate and automatic organ segmentation is critical for computer-aided analysis towards clinical decision support and treatment planning. State-of-the-art approaches have achieved remarkable segmentation accuracy on large organs, such as the liver and kidneys. However, most of these methods do not perform well on small organs, such as the pancreas, gallbladder, and adrenal glands, especially when lacking sufficient training data. This paper presents an automatic approach for small organ segmentation with limited training data using two cascaded steps—localization and segmentation. The localization stage involves the extraction of the region of interest after the registration of images to a common template and during the segmentation stage, a voxel-wise label map of the extracted region of interest is obtained and then transformed back to the original space. In the localization step, we propose to utilize a graph-based groupwise image registration method to build the template for registration so as to minimize the potential bias and avoid getting a fuzzy template. More importantly, a novel knowledge-aided convolutional neural network is proposed to improve segmentation accuracy in the second stage. This proposed network is flexible and can combine the effort of both deep learning and traditional methods, consequently achieving better segmentation relative to either of individual methods. The ISBI 2015 VISCERAL challenge dataset is used to evaluate the presented approach. Experimental results demonstrate that the proposed method outperforms cutting-edge deep learning approaches, traditional forest-based approaches, and multi-atlas approaches in the segmentation of small organs.

For many segmentation tasks, especially for the biomedical image, the topological prior is vital information which is useful to exploit. The containment/nesting is a typical inter-class geometric relationship. In the MICCAI Brain tumor segmentation challenge, with its three hierarchically nested classes ‘whole tumor’, ‘tumor core’, ‘active tumor’, the nested classes relationship is introduced into the 3D-residual-Unet architecture. The network comprises a context aggregation pathway and a localization pathway, which encodes increasingly abstract representation of the input as going deeper into the network, and then recombines these representations with shallower features to precisely localize the interest domain via a localization path. The nested-class-prior is combined by proposing the multi-class activation function and its corresponding loss function. The model is trained on the training dataset of Brats2018, and 20% of the dataset is regarded as the validation dataset to determine parameters. When the parameters are fixed, we retrain the model on the whole training dataset. The performance achieved on the validation leaderboard is 86%, 77% and 72% Dice scores for the whole tumor, enhancing tumor and tumor core classes without relying on ensembles or complicated post-processing steps. Based on the same start-of-the-art network architecture, the accuracy of nested-class (enhancing tumor) is reasonably improved from 69% to 72% compared with the traditional Softmax-based method which blind to topological prior.

For many biological image segmentation tasks, including topological knowledge, such as the nesting of classes, can greatly improve results. However, most ‘out-of-the-box’ CNN models are still blind to such prior information. In this paper, we propose a novel approach to encode this information, through a multi-level activation layer and three compatible losses. We benchmark all of them on nuclei segmentation in bright-field microscopy cell images from the 2018 Data Science Bowl challenge, offering an exemplary segmentation task with cells and nested subcellular structures. Our scheme greatly speeds up learning, and outperforms standard multi-class classification with soft-max activation and a previously proposed method stemming from it, improving the Dice score significantly (p-values \(

The assessment of bone lesion is crucial for the diagnostic and therapeutic planning of multiple myeloma (MM). \(^{68}\)Ga-Pentixafor PET/CT can capture the abnormal molecular expression of CXCR-4 in addition to anatomical changes. However, the whole-body detection of dozens of lesions on hybrid imaging is tedious and error-prone. In this paper, we adopt a cascaded convolutional neural networks (CNN) to form a W-shaped architecture (W-Net). This deep learning method leverages multimodal information for lesion detection. The first part of W-Net extracts skeleton from CT scan and the second part detect and segment lesions. The network was tested on 12 \(^{68}\)Ga-Pentixafor PET/CT scans of MM patients using 3-folder cross validation. The preliminary results showed that W-Net can automatically learn features from multimodal imaging for MM bone lesion detection. The proof-of-concept study encouraged further development of deep learning approach for MM lesion detection with increased number of subjects.