Search

Time-resolved volumetric magnetic resonance imaging (4D MRI) could be used to address organ motion in image-guidedinterventions like tumor ablation. Current 4D reconstruction techniques are unsuitable for most interventional settings becausethey are limited to specific breathing phases, lack temporal/spatial resolution, and have long prior acquisitions or reconstructiontimes. Deep learning-based (DL) 4D MRI approaches promise to overcome these shortcomings but are sensitive to anotherproblem, which is domain shift. This work shows that transfer learning (TL) can help alleviate this key challenge. Data from20 healthy subjects were acquired and split into 16 source and 4 target domain subjects. We compare models trained fromscratch on target domain data with models fine-tuned from a pre-trained model. Significant improvements (P

Background and Objective: Ultrasound (US) devices are often used in percutanous interventions. Due to their low image quality, the US image slices are aligned with pre-operative Computed Tomography/Magnetic Resonance Imaging (CT/MRI) images to enable better visibilities of anatomies during the intervention. This work aims at improving the deep learning one shot registration by using less loops through deep learning networks.Methods: We propose two cascade networks which aim at improving registration accuracy by less loops. The InitNet-Regression-LoopNet (IRL) network applies the plane regression method to detect the orientation of the predicted plane derived from the previous loop, then corrects input CT/MRI volume orientation and improves the prediction iteratively. The InitNet-LoopNet-MultiChannel (ILM) comprises two cascade networks, where an InitNet is trained with low resolution images toperform coarse registration. Then, a LoopNet wraps the high resolution images and result of the previous loop into a three channel input and trained to improve prediction accuracy in every loop. Results: We benchmark the two cascade networks on 1035 clinical images from 52 patients , yielding an improved registration accuracy with LoopNet. The IRL achieved an average angle error of 13.3° and an average distance error of 4.5 millimieter. It out-performs the ILM network with angle error 17.4° and distance error 4.9 millimeter and the InitNet with angle error 18.6° and distance error 4.9 millimeter. Our results show the efficiency of the proposed registration networks, which have the potential to improve the robustness and accuracy of intraoperative patient registration.

Preoperative assessment of the proximity of critical structures to the tumors is crucial in avoiding unnecessary damage during prostate cancer treatment. A patient-specific 3D anatomical model of those structures, namely the neurovascular bundles (NVB) and the external urethral sphincters (EUS), can enable physicians to perform such assessments intuitively. As a crucial step to generate a patient-specific anatomical model from preoperative MRI in a clinical routine, we propose a multi-class automatic segmentation based on an anisotropic convolutional network. Our specific challenge is to train the network model on a unique source dataset only available at a single clinical site and deploy it to another target site without sharing the original images or labels. As network models trained on data from a single source suffer from quality loss due to the domain shift, we propose a semi-supervised domain adaptation (DA) method to refine the model’s performance in the target domain. Our DA method combines transfer learning and uncertainty guided self-learning based on deep ensembles. Experiments on the segmentation of the prostate, NVB, and EUS, show significant performance gain with the combination of those techniques compared to pure TL and the combination of TL with simple self-learning ($${p} p < 0.005 for all structures using a Wilcoxon’s signed-rank test). Results on a different task and data (Pancreas CT segmentation) demonstrate our method’s generic application capabilities. Our method has the advantage that it does not require any further data from the source domain, unlike the majority of recent domain adaptation strategies. This makes our method suitable for clinical applications, where the sharing of patient data is restricted.

Recent studies demonstrated the eligibility of convolutional neural networks (CNNs) for solving the image registration problem. CNNs enable faster transformation estimation and greater generalization capability needed for better support during medical interventions. Conventional fully-supervised training requires a lot of high-quality ground truth data such as voxel-to-voxel transformations, which typically are attained in a too tedious and error-prone manner. In our work, we use weakly-supervised learning, which optimizes the model indirectly only via segmentation masks that are a more accessible ground truth than the deformation fields. Concerning the weak supervision, we investigate two segmentation similarity measures: multiscale Dice similarity coefficient (mDSC) and the similarity between segmentation-derived signed distance maps (SDMs). We show that the combination of mDSC and SDM similarity measures results in a more accurate and natural transformation pattern together with a stronger gradient coverage. Furthermore, we introduce an auxiliary input to the neural network for the prior information about the prostate location in the MR sequence, which mostly is available preoperatively. This approach significantly outperforms the standard two-input models. With weakly labelled MR-TRUS prostate data, we showed registration quality comparable to the state-of-the-art deep learning-based method.

Accurate and reliable segmentation of the prostate gland in MR images can support the clinical assessment of prostate cancer, as well as the planning and monitoring of focal and loco-regional therapeutic interventions. Despite the availability of multi-planar MR scans due to standardized protocols, the majority of segmentation approaches presented in the literature consider the axial scans only. In this work, we investigate whether a neural network processing anisotropic multi-planar images could work in the context of a semantic segmentation task, and if so, how this additional information would improve the segmentation quality.We propose an anisotropic 3D multi-stream CNN architecture, which processes additional scan directions to produce a high-resolution isotropic prostate segmentation. We investigate two variants of our architecture, which work on two (dual-plane) and three (triple-plane) image orientations, respectively. The influence of additional information used by these models is evaluated by comparing them with a single-plane baseline processing only axial images. To realize a fair comparison, we employ a hyperparameter optimization strategy to select optimal configurations for the individual approaches.Training and evaluation on two datasets spanning multiple sites show statistical significant improvement over the plain axial segmentation (p0.05 on the Dice similarity coefficient). The improvement can be observed especially at the base (0.898 single-plane vs. 0.906 triple-plane) and apex (0.888 single-plane vs. 0.901 dual-plane).This study indicates that models employing two or three scan directions are superior to plain axial segmentation. The knowledge of precise boundaries of the prostate is crucial for the conservation of risk structures. Thus, the proposed models have the potential to improve the outcome of prostate cancer diagnosis and therapies.

Background and Objective Liver tumor ablation is often guided by ultrasound (US). Due to poor image quality, intraoperative US is fused with preoperative computed tomography or magnetic tomography (CT/MR) images to provide visual guidance. As of today, the underlying 2D US to 3D CT/MR registration problem remains a very challenging task. Methods We propose a novel pipeline to address this registration problem. Contrary to previous work, we do not formulate the problem as a regression task, which - for the given registration problem - achieves a low performance regarding accuracy and robustness due to the limited US soft-tissue contrast and the inter-patient variability on liver vessels. Instead, we first estimate the US probe angle roughly by using a classification network. Given this coarse initialization, we then improve the registration by formulation of the problem as a segmentation task, estimating the US plane in the 3D CT/MR through segmentation. Results We benchmark our approach on 1035 clinical images from 52 patients, yielding average registration errors of 11.6 ° and 4.7 mm, which outperforms the state of the art SVR method[1]. Conclusion Our results show the efficiency of the proposed registration pipeline, which has potential to improve the robustness and accuracy of intraoperative patient registration.

In interstitial high-dose rate brachytherapy, liver cancer is treated by internal radiation, requiring percutaneous placement of applicators within or close to the tumor. To maximize utility, the optimal applicator configuration is pre-planned on magnetic resonance images. The pre-planned configuration is then implemented via a magnetic resonance-guided intervention. Mapping the pre-planning information onto interventional data would reduce the radiologist’s cognitive load during the intervention and could possibly minimize discrepancies between optimally pre-planned and actually placed applicators. We propose a fast and robust two-step registration framework suitable for interventional settings: first, we utilize a multi-resolution rigid registration to correct for differences in patient positioning (rotation and translation). Second, we employ a novel iterative approach alternating between bias field correction and Markov random field deformable registration in a multi-resolution framework to compensate for non-rigid movements of the liver, the tumors and the organs at risk. In contrast to existing pre-correction methods, our multi-resolution scheme can recover bias field artifacts of different extents at marginal computational costs. We compared our approach to deformable registration via B-splines, demons and the SyN method on 22 registration tasks from eleven patients. Results showed that our approach is more accurate than the contenders for liver as well as for tumor tissues. We yield average liver volume overlaps of 94.0 ± 2.7% and average surface-to-surface distances of 2.02 ± 0.87 mm and 3.55 ± 2.19 mm for liver and tumor tissue, respectively. The reported distances are close to (or even below) the slice spacing (2.5 – 3.0 mm) of our data. Our approach is also the fastest, taking 35.8 ± 12.8 s per task. The presented approach is sufficiently accurate to map information available from brachytherapy pre-planning onto interventional data. It is also reasonably fast, providing a starting point for computer-aidance during intervention.

The extraction of spines from medical records in a fast yet accurate way is a challenging task, especially for large data sets. Addressing this issue, we present a framework based on convolutional neural networks for the reconstruction of the spinal shape and curvature, making statistical assessments feasible on epidemiological scale. Our method uses a two-step strategy. First, anchor vertebrae and the spinal centerline in between them get extracted. Second, the centerlines are transformed into a common coordinate system to enable comparisons and statistical assessments across subjects. Our networks were trained on 103 subjects, where we achieved accuracies of 3.3 mm on average, taking at most 1 s per record, which eases the handling of even very large cohorts. Without any further training, we validated our model on study data of about 3400 subjects with only 10 cases of failure, which demonstrates the robustness of our method with respect to the natural variability in spinal shape and curvature. A thorough statistical analysis of the results underpins the importance of our work. Specifically, we show that the spinal curvature is significantly influenced by the body mass index of a subject. Moreover, we show that the same findings arise when Cobb angles are considered instead of direct curvature measures. To this end, we propose a generalization of classical Cobb angles that can be evaluated algorithmically and can also serve as a useful (visual) tool for physicians in everyday clinical practice.

In the last decades, the increasing medical interest in magnetic resonance imaging (MRI) of the spine gave rise to a growing number of publications on computerized methods for spine analysis, covering goals such as localization and segmentation of vertebrae and intervertebral discs as well as the extraction and segmentation of the spinal canal and cord. We provide a critical systematic review to work in the field, putting focus on approaches that can be applied to different imaging sequences and settings. Work is analysed on two levels. First, methods are reviewed in detail so that the reader understands justifications and constraints of particular work. Second, work is classified according to relevant attributes and tabulated to give an impression on recent trends. We discuss the general methodical and evaluational aspects of the work as well as challenges specific to MRI such as the lack of intensity standardization and partial volume effects. Methods can be condensed to a small number of optimization frameworks, e.g., graphical models, cost-minimal paths and deformable models. Works sharing the same framework mainly differentiate by the types of information, i.e., pose, geometry and appearance, that are used and by the implementation thereof. MRI-specific challenges are rarely addressed explicitly, calling into question the applicability of most methods to changing imaging sequences or settings. Most often, little attention is paid to evaluation, meaning that results lack comparability and reproducibility although publicly available data sets exist. The diversity of MRI sequences and settings still poses challenges to computerized spine analysis. Further research is necessary to implement methods that are actually applicable in practice, e.g., in clinical routine or for study purposes. Certainly, manual guidance will be necessary at some point, for instance to deal with changing subject positions. Therefore, future work should put attention to the appropriate integration of manual interaction.

Background and Objective: We propose an automatic approach for fast vertebral body segmentation in three-dimensional magnetic resonance images of the whole spine. Previous works are limited to the lower thoracolumbar section and often take minutes to compute, which is problematic in clinical routine, for study data sets with numerous subjects or when the cervical or upper thoracic spine is to be analyzed. Methods: We address these limitations by a novel graph cut formulation based on vertebra patches extracted along the spine. For each patch, our formulation incorporates appearance and shape information derived from a task-specific convolutional neural network as well as star-convexity constraints that ensure a topologically correct segmentation of each vertebra. When segmenting vertebrae individually, ambiguities will occur due to overlapping segmentations of adjacent vertebrae. We tackle this problem by novel non-overlap constraints between neighboring patches based on so-called encoding swaps. The latter allow us to obtain a globally optimal multi-label segmentation of all vertebrae in polynomial time. Results: We validated our approach on two data sets. The first contains T1- and T2-weighted whole spine images of 64 subjects with varying health conditions. The second comprises 23 T2-weighted thoracolumbar images of young healthy adults and is publicly available. Our method yielded Dice coefficients of 93.8 ± 2.6% and 96.0 ± 1.0% for both data sets with a run time of 1.35 ± 0.08 s and 0.90 ± 0.03 s per vertebra on consumer hardware. A complete whole spine segmentation took 32.4 ± 1.92 s on average. Conclusions: Our results are superior to those of previous works at a fraction of their run time, which illustrates the efficiency and effectiveness of our whole spine segmentation approach.

Individualized and accurate segmentations of the prostate are essential for diagnosis as well as therapy planning in prostate cancer (PCa). Most of the previously proposed prostate segmentation approaches rely purely on axial MRI scans, which suffer from low out-of-plane resolution. We propose a method that makes use of sagittal and coronal MRI scans to improve the accuracy of segmentation. These scans are typically acquired as standard of care for PCa staging, but are generally ignored by the segmentation algorithms. Our method is based on a multi-stream 3D convolutional neural network for the automatic extraction of isotropic high resolution segmentations from MR images. We evaluated segmentation performance on an isotropic high resolution ground truth (n = 40 subjects). The results show that the use of multi-planar volumes for prostate segmentation leads to improved segmentation results not only for the whole prostate (92.1% Dice similarity coefficient), but also in apex and base regions.

A rising number of epidemiological studies apply imaging technologies. Images are not features themselves but provide raw data from which features are extracted. Different to other applications of analysis of medical images the data is analyzed statistically across the cohort. It results in unique requirements regarding the development of methods to efficiently integrate varying domain knowledge into the process without compromising comparability of results across subjects or time. Examples from two different strategies are presented and discussed.

We propose a semi-automatic approach for aorta centerline extraction in contrast-enhanced MRI, making aorta length analysis feasible on large scale. Starting from user-specified start and end regions, we extract the aorta path in between the regions automatically. The extraction is formulated as an optimization problem, seeking for the path that most likely runs central to the aorta. To this end, we exploit that the aorta distinguishes from the surrounding by strong image gradients that point inwards to the aorta’s center due to contrast-enhanced imaging. We also include additional means of manual guidance to resolveerroneous cases. Experiments on data of 19 subjects yielded results that are close to the inter-reader variability. The average distance to the ground truth was 1.89 ± 1.54 mm, while aorta lengths deviated by only 0.66 ± 0.49 %.

Diagnosis of neuromuscular diseases in ultrasonography is a challenging task since experts are often unable to discriminate between healthy and pathological cases. A computer-aided diagnosis (CAD) system for skeletal muscle ultrasonography was developed and tested for myositis detection in ultrasound images of biceps brachii. Several types of features were extracted from rectangular and polygonal image regions-of-interest (ROIs), including first-order statistics, wavelet-based features, and Haralick’s features. Features were chosen that are sensitive to the change in contrast and structure for pathological ultrasound images of neuromuscular diseases. The number of features was reduced by applying different sequential feature selection strategies followed by a supervised principal component analysis. For classification, two linear approaches were investigated: Fisher’s classifier and the linear support vector machine (SVM) as well as the nonlinear $$k$$ -nearest neighbor approach. The CAD system was benchmarked on datasets of 18 subjects, seven of which were healthy, while 11 were affected by myositis. Three expert radiologists provided pre-classification and testing interpretations. Leave-one-out cross-validation on the training data revealed that the linear SVM was best suited for discriminating healthy and pathological muscle tissue, achieving 85/87 % accuracy, 90 % sensitivity, and 83/85 % specificity, depending on the radiologist. A muscle ultrasonography CAD system was developed, allowing a classification of an ultrasound image by one-click positioning of rectangular ROIs with minimal user effort. The applicability of the system was demonstrated with the challenging example of myositis detection, showing highly accurate results that were robust to imprecise user input.

We tackle the problem of unsupervised object cosegmentation combining automatic image selection, cosegmentation, and knowledge transfer to yet unlabelled images. Furthermore, we overcome the limitations often present in state-of-the-art methods in object cosegmentation, namely, high complexity and poor scalability w.r.t. image set size. Our proposed approach is robust, reasonably fast, and scales linearly w.r.t. the image set size. We tested our approach on two commonly used cosegmentation data sets and outperformed some of the state-of-the-art methods using significantly less information than possible. Additionally, results indicate the applicability of our approach on larger image sets.

We propose an automatic approach to aorta segmentation in axial cardiac cine MRI. The segmentation task is formulated as a prob- abilistic inference problem, seeking for the most probable constellation of aorta locations and shapes in time. To this end, a graphical model is developed that implements the mutual dependencies of the aorta param- eters along the cine sequence. Our approach integrates effective means of manual guidance for post-correction in case of erroneous results, re- quiring only user interaction where necessary. Experiments on a data set of 20 cine sequences showed average Dice coefficients close to the inter- reader variability while outperforming previous work in the field. Only two post-corrections were required for the entire data set. Results also indicate high stability of our approach w.r.t. re-parameterization.

In recent years, analysis of magnetic resonance images of the spine gained considerable interest with vertebra localization being a key step for higher level analysis. Approaches based on trained appearance - which are de facto standard - may be inappropriate for certain tasks, because processing usually takes several minutes or training data is unavailable. Learning-free approaches have yet to show there competitiveness for whole-spine localization. Our work fills this gap. We combine a fast engineered detector with a novel vertebrae appearance similarity concept. The latter can compete with trained appearance, which we show on a data set of 64 \(T_1\)- and 64 \(T_2\)-weighted images. Our detection took \(27.7 \pm 3.78\) s with a detection rate of 96.0 % and a distance to ground truth of \(3.45 \pm 2.2\) mm, which is well below the slice thickness.

Identifying differences among the distribution of samples of different observations is an important issue in many research fields. We provide a general framework to detect these difference spots in d-dimensional feature space. Such spots occur not only at various locations, they may also come in various shapes and multiple sizes, even at the same location. We address these challenges by a scale-space representation of the density function difference of the observations in feature space. Using three classification scenarios from UCI Machine Learning Repository we show that interest spots carry valuable information about a data set. To this end, we establish a simple decision rule on top of our framework. Results indicate state-of-the-art performance, underpinning the importance of the information that is carried by the detected spots. Furthermore, we outline that the output of our framework can be used to guide exploratory visualization of high-dimensional feature spaces.

Identifying differences among the sample distributions of different observations is an important issue in many fields ranging from medicine over biology and chemistry to physics. We address this issue, providing a general framework to detect difference spots of interest in feature space. Such spots occur not only at various locations, they may also come in various shapes and multiple sizes, even at the same location. We deal with these challenges in a scale-space detection framework based on the density function difference of the observations. Our framework is intended for semi-automatic processing, providing human-interpretable interest spots for further investigation of some kind, e.g., for generating hypotheses about the observations. Such interest spots carry valuable information, which we outline at a number of classification scenarios from UCI Machine Learning Repository; namely, classification of benign/malign breast cancer, genuine/forged money and normal/spondylolisthetic/disc-herniated vertebral columns. To this end, we establish a simple decision rule on top of our framework, which bases on the detected spots. Results indicate state-of-the-art classification performance, which underpins the importance of the information that is carried by these interest spots.

Epidemiology aims to provide insight into disease causations. Hence, subject groups (cohorts) are analyzed to correlate the subjects’ varying lifestyles, their medical properties and diseases. Recently, these cohort studies comprise medical image data. We assess potential relations between image-derived variables of the lumbar spine with lower back pain in a cross-sectional study. Therefore, an Interactive Visual Analysis (IVA) framework was created and tested with 2,540 segmented lumbar spine data sets. The segmentation results are evaluated and quantified by employing shape-describing variables, such as spine canal curvature and torsion. We analyze mutual dependencies among shape-describing variables and non-image variables, e.g., pain indicators. Therefore, we automatically train a decision tree classifier for each non-image variable. We provide an IVA technique to compare classifiers with a decision tree quality plot. As a first result, we conclude that image-based variables are only sufficient to describe lifestyle factors within the data. A correlation between lumbar spine shape and lower back pain could not be found with the automatically trained classifiers. However, the presented approach is a valuable extension for the IVA of epidemiological data. Hence, relations between non-image variables were successfully detected and described.

We address the task of aortic diameter measurement in (noncontrast- enhanced) plain axial cardiac cine MRI. To this end, we set up a likelihood maximization problem which allows us to recover globally optimal aorta locations and diameters of the cine sequence efficiently. Our approach provides intuitive means of manual post-correction and requires little user interaction, making large-scale image analysis feasible. Experiments on a data set of 20 cine sequences with 30 time frames showed (at least) pixel-accurate diameter measurements which are also highly stable against re-parameterization.

Background Object detection in 3-D medical images is often necessary for constraining a segmentation or registration task. It may be a task in its own right as well, when instances of a structure, e.g. the lymph nodes, are searched. Problems from occlusion, illumination and projection do not arise, making the problem simpler than object detection in photographies. However, objects of interest are often not well contrasted against the background. Influence from noise and other artifacts is much stronger and shape and appearance may vary substantially within a class. Methods Deformable models capture the characteristic shape of an anatomic object and use constrained deformation for hypothesing object boundaries in image regions of low or non-existing contrast. Learning these constraints requires a large sample data base. We show that training may be replaced by readily available user knowledge defining a prototypical deformable part model. If structures have a strong part-relationship, or if they may be found based on spatially related guiding structures, or if the deformation is rather restricted, the supporting data information suffices for solving the detection task. We use a finite element model to represent anatomic variation by elastic deformation. Complex shape variation may be represented by a hierarchical model with simpler part variation. The hierarchy may be represented explicitly as a hierarchy of sub-shapes, or implicitly by a single integrated model. Data support and model deformation of the complete model can be represented by an energy term, serving as quality-of-fit function for object detection. Results The model was applied to detection and segmentation tasks in various medical applications in 2- and 3-D scenes. It has been shown that model fitting and object detection can be carried out efficiently by a combination of a local and global search strategy using models that are parameterized for the different tasks. Conclusions A part-based elastic model represents complex within-class object variation without training. The hierarchy of parts may specify relationship to neighboring anatomical objects in object detection or a part-decomposition of a complex anatomic structure. The intuitive way to incorporate domain knowledge has a high potential to serve as easily adaptable method to a wide range of different detection tasks in medical image analysis.

Muscle ultrasonography is a convenient technique to visualize healthy and pathological muscle tissue as it is non-invasive and image acquisition can be done in real-time. In this paper, a texture-based approach is presented to detect myositis in ultrasound images automatically. We compute different texture features like wavelet transform features and first-order grey-level intensity statistics of a relevant central image patch carrying structure and intensity information of muscle tissue. Using a combination of these information we reached an accuracy of classification of 92.20 % with our approach on a training data set of 63 clinically pre-classified data sets.

Convex quadratic optimization is one of the most widely used concepts in image segmentation. It facilitates a wide range of information sources, such as edge, intensity, texture and shape. The problem is especially challenging for the multi-label case, even being NP-hard in its most general setting. Therefore, fast "solutions", as the α-expansion of [1], are limited to local optimality. Addressing this problem, several approaches relax the labeling integrality condition, resulting in quadratic programs (QPs) like in [2] and in [4], which can be solved in polynomial time. Although this is efficient in a theoretical sense, large-scale QPs that arise from typical multi-label tasks can rarely be used for image segmentation directly due to either time or space constraints, or both. We address this issue by an adaptive domain subdivision scheme, reducing the problem to a short sequence of spatially smoothed medium-scale QPs, which subsequently better approximate the large-scale program. Our scheme is globally optimal in terms of the approximated problem. Putting our main focus on the subdivision, we restrict ourselves to minimization of the popular but rather simple piecewise constant MumfordShah functional. Therefore, we seek for a labeling that trades off the length of the labeling border and the approximation of image intensity u by known reference intensitites ui for each label i. For discrete domains the associated energy can be written as

In recent years, image-guided thermal ablations have become a considerable treatment method for cancer patients, including support through navigational systems. One of the most critical challenges in these systems is the registration between the intraoperative images and the preoperative volume. The motion secondary to inspiration makes registration even more difficult. In this work, we propose a coarse-fine fast patient registration technique to solve the problem of motion compensation. In contrast to other state-of-the-art methods, we focus on improving the convergence range of registration. To this end, we make use of a Deep Learning 2D U-Net framework to extract the vessels and liver borders from intraoperative ultrasound images and employ the segmentation results as regions of interest in the registration. After an initial 3D-3D registration during breath hold, the following motion compensation is achieved using a 2D-3D registration. Our approach yields a convergence rate of over 70% with an accuracy of $1.97 \pm 1.07$ mm regarding the target registration error. The 2D-3D registration is GPU-accelerated with a time cost of less than 200 ms.