Search

Purpose: Pelvic bone segmentation in CT has always been an essential step in clinical diagnosis and surgery planning of pelvic bone diseases. Existing methods for pelvic bone segmentation are either hand-crafted or semi-automatic and achieve limited accuracy when dealing with image appearance variations due to the multi-site domain shift, the presence of contrasted vessels, coprolith and chyme, bone fractures, low dose, metal artifacts, etc. Due to the lack of a large-scale pelvic CT dataset with annotations, deep learning methods are not fully explored., Methods: In this paper, we aim to bridge the data gap by curating a large pelvic CT dataset pooled from multiple sources, including 1184 CT volumes with a variety of appearance variations. Then, we propose for the first time, to the best of our knowledge, to learn a deep multi-class network for segmenting lumbar spine, sacrum, left hip, and right hip, from multiple-domain images simultaneously to obtain more effective and robust feature representations. Finally, we introduce a post-processor based on the signed distance function (SDF)., Results: Extensive experiments on our dataset demonstrate the effectiveness of our automatic method, achieving an average Dice of 0.987 for a metal-free volume. SDF post-processor yields a decrease of 15.1% in Hausdorff distance compared with traditional post-processor., Conclusion: We believe this large-scale dataset will promote the development of the whole community and open source the images, annotations, codes, and trained baseline models at https://github.com/ICT-MIRACLE-lab/CTPelvic1K .

Patient-specific orthopedic knee surgery planning requires precisely segmenting from 3D CT images multiple knee bones, namely femur, tibia, fibula, and patella, around the knee joint with severe pathologies. In this work, we propose a fully automated, highly precise, and computationally efficient segmentation approach for multiple bones. First, each bone is initially segmented using a model-based marginal space learning framework for pose estimation followed by non-rigid boundary deformation. To recover shape details, we then refine the bone segmentation using graph cut that incorporates the shape priors derived from the initial segmentation. Finally we remove overlap between neighboring bones using multi-layer graph partition. In experiments, we achieve simultaneous segmentation of femur, tibia, patella, and fibula with an overall accuracy of less than 1mm surface-to-surface error in less than 90s on hundreds of 3D CT scans with pathological knee joints.

The diversity in appearance of diseased lung tissue makes automatic segmentation of lungs from CT with severe pathologies challenging. To overcome this challenge, we rely on contextual constraints from neighboring anatomies to detect and segment lung tissue across a variety of pathologies. We propose an algorithm that combines statistical learning with these anatomical constraints to seek a segmentation of the lung consistent with adjacent structures, such as the heart, liver, spleen, and ribs. We demonstrate that our algorithm reduces the number of failed detections and increases the accuracy of the segmentation on unseen test cases with severe pathologies.

Ultrasound acquisition is a challenging task that requires simultaneous adjustment of several acquisition parameters (the depth, the focus, the frequency and its operation mode). If the acquisition parameters are not properly chosen, the resulting image will have a poor quality and will degrade the patient diagnosis and treatment workflow. Several hardware-based systems for autotuning the acquisition parameters have been previously proposed, but these solutions were largely abandoned because they failed to properly account for tissue inhomogeneity and other patient-specific characteristics. Consequently, in routine practice the clinician either uses population-based parameter presets or manually adjusts the acquisition parameters for each patient during the scan. In this paper, we revisit the problem of autotuning the acquisition parameters by taking a completely novel approach and producing a solution based on image analytics. Our solution is inspired by the autofocus capability of conventional digital cameras, but is significantly more challenging because the number of acquisition parameters is large and the determination of "good quality" images is more difficult to assess. Surprisingly, we show that the set of acquisition parameters which produce images that are favored by clinicians comprise a 1D manifold, allowing for a real-time optimization to maximize image quality. We demonstrate our method for acquisition parameter autotuning on several live patients, showing that our system can start with a poor initial set of parameters and automatically optimize the parameters to produce high quality images.

This paper proposes a fully automatic approach for computing Nuchal Translucency (NT) measurement in an ultrasound scans of the mid-sagittal plane of a fetal head. This is an improvement upon current NT measurement methods which require manual placement of NT measurement points or user-guidance in semi-automatic segmentation of the NT region. The algorithm starts by finding the pose of the fetal head using discriminative learning-based detectors. The fetal head serves as a robust anchoring structure and the NT region is estimated from the statistical relationship between the fetal head and the NT region. Next, the pose of the NT region is locally refined and its inner and outer edge approximately determined via Dijkstra's shortest path applied on the edge-enhanced image. Finally, these two region edges are used to define foreground and background seeds for accurate graph cut segmentation. The NT measurement is computed from the segmented region. Experiments show that the algorithm efficiently and effectively detects the NT region and provides accurate NT measurement which suggests suitability for clinical use.

We present a two-stage method for effective and efficient detection of one or multiple anatomical landmarks in an arbitrary 3D volume. The first stage of nearest neighbor matching is to roughly estimate the landmark locations. It searches out of 100,000 volumes for the closest to an input volume and then transfers landmark annotations to the input. The second stage of submodular optimization is to refine the landmark locations by running discriminative landmark detectors within the search ranges constrained by the first stage results. Further it coordinates multiple detectors with a search strategy optimized on the fly to reduce the overall computation cost arising in a submodular formulation. We validate the accuracy, speed and robustness of our approach by detecting body regions and landmarks in a dataset of 2500 CT scans.

In this paper, we present a novel method by incorporating information theory into the learning-based approach for automatic and accurate pelvic organ segmentation (including the prostate, bladder and rectum). We target 3D CT volumes that are generated using different scanning protocols (e.g., contrast and non-contrast, with and without implant in the prostate, various resolution and position), and the volumes come from largely diverse sources (e.g., diseased in different organs). Three key ingredients are combined to solve this challenging segmentation problem. First, marginal space learning (MSL) is applied to efficiently and effectively localize the multiple organs in the largely diverse CT volumes. Second, learning techniques, steerable features, are applied for robust boundary detection. This enables handling of highly heterogeneous texture pattern. Third, a novel information theoretic scheme is incorporated into the boundary inference process. The incorporation of the Jensen-Shannon divergence further drives the mesh to the best fit of the image, thus improves the segmentation performance. The proposed approach is tested on a challenging dataset containing 188 volumes from diverse sources. Our approach not only produces excellent segmentation accuracy, but also runs about eighty times faster than previous state-of-the-art solutions. The proposed method can be applied to CT images to provide visual guidance to physicians during the computer-aided diagnosis, treatment planning and image-guided radiotherapy to treat cancers in pelvic region.

Cardiac computed tomography (CT) is the primary noninvasive imaging modality to diagnose coronary artery disease. Though various methods have been proposed for coronary artery segmentation, most rely on at least one user click to provide a seed point for initialization. Automatic detection of the coronary ostia (where coronaries originate from the aorta), including both the native coronary ostia and graft ostia of the bypass coronaries, can make the whole coronary exam workflow fully automatic, therefore increasing a physician's throughput. Anatomical structures (native coronary ostia) and pathological structures (graft ostia) often require significantly different detection methods. The native coronary ostia are well constrained by the surrounding structures, therefore are detected as a global object. Detecting the graft ostia is far more difficult due to the large variation in graft position. A new searching strategy is proposed to efficiently guide the focus of analysis and, at the same time, reduce the false positive detections. Since the bypass coronaries are grafted on the ascending aorta surface, the ascending aorta is first segmented to constrain the search. The quantitative prior distribution of the graft ostia on the aorta surface is learned from a training set to significantly reduce the searching space further. Efficient local image features are extracted around each candidate point on the aorta surface to train a detector. The proposed method is computationally efficient, taking about 0.40 seconds to detect both native and graft ostia in a volume with around 512 x 512 x 200 voxels.

We propose an automatic algorithm for phase labeling that relies on the intensity changes in anatomical regions due to the contrast agent propagation. The regions (specified by aorta, vena cava, liver, and kidneys) are first detected by a robust learning-based discriminative algorithm. The intensities inside each region are then used in multi-class LogitBoost classifiers to independently estimate the contrast phase. Each classifier forms a node in a decision tree which is used to obtain the final phase label. Combining independent classification from multiple regions in a tree has the advantage when one of the region detectors fail or when the phase training example database is imbalanced. We show on a dataset of 1016 volumes that the system correctly classifies native phase in 96.2% of the cases, hepatic dominant phase (92.2%), hepatic venous phase (96.7%), and equilibrium phase (86.4%) in 7 seconds on average.

Recently conducted clinical studies prove the utility of Coronary Computed Tomography Angiography (CCTA) as a viable alternative to invasive angiography for the detection of Coronary Artery Disease (CAD). This has lead to the development of several algorithms for automatic detection and grading of coronary stenoses. However, most of these methods focus on detecting calcified plaques only. A few methods that can also detect and grade non-calcified plaques require substantial user involvement. In this paper, we propose a fast and fully automatic system that is capable of detecting, grading and classifying coronary stenoses in CCTA caused by all types of plaques. We propose a four-step approach including a learning-based centerline verification step and a lumen cross-section estimation step using random regression forests. We show state-of-the-art performance of our method in experiments conducted on a set of 229 CCTA volumes. With an average processing time of 1.8 seconds per case after centerline extraction, our method is significantly faster than competing approaches.

As a minimally invasive surgery to treat left atrial (LA) fibrillation, catheter based ablation uses high radio-frequency energy to eliminate potential sources of the abnormal electrical events, especially around the ostia of pulmonary veins (PV). Due to large structural variations of the PV drainage pattern, a personalized LA model is helpful to translate a generic ablation strategy to a specific patient's anatomy. Overlaying the LA model onto 2D fluoroscopic images provides valuable visual guidance during surgery. A holistic shape model is not accurate enough to represent the whole shape population of the LA. In this paper, we propose a part based LA model (including the chamber, appendage, and four major PVs) and each part is a much simpler anatomical structure compared to the holistic one. Our approach works on un-gated C-arm CT, where thin boundaries between the LA blood pool and surrounding tissues are often blurred due to the cardiac motion artifacts (which presents a big challenge compared to the highly contrasted gated CT/MRI). To avoid segmentation leakage, the shape prior is exploited in a model based approach to segment the LA parts. However, independent detection of each part is not optimal and its robustness needs further improvement (especially for the appendage and PVs). We propose to enforce a statistical shape constraint during the estimation of pose parameters (position, orientation, and size) of different parts. Our approach is computationally efficient, taking about 1.5 s to process a volume with 256 x 256 x 250 voxels. Experiments on 469 C-arm CT datasets demonstrate its robustness.

Simple algorithms for segmenting healthy lung parenchyma in CT are unable to deal with high density tissue common in pulmonary diseases. To overcome this problem, we propose a multi-stage learning-based approach that combines anatomical information to predict an initialization of a statistical shape model of the lungs. The initialization first detects the carina of the trachea, and uses this to detect a set of automatically selected stable landmarks on regions near the lung (e.g., ribs, spine). These landmarks are used to align the shape model, which is then refined through boundary detection to obtain fine-grained segmentation. Robustness is obtained through hierarchical use of discriminative classifiers that are trained on a range of manually annotated data of diseased and healthy lungs. We demonstrate fast detection (35s per volume on average) and segmentation of 2 mm accuracy on challenging data.

We present a novel generic segmentation system for the fully automatic multi-organ segmentation from CT medical images. Thereby we combine the advantages of learning-based approaches on point cloud-based shape representation, such a speed, robustness, point correspondences, with those of PDE-optimization-based level set approaches, such as high accuracy and the straightforward prevention of segment overlaps. In a benchmark on 10-100 annotated datasets for the liver, the lungs, and the kidneys we show that the proposed system yields segmentation accuracies of 1.17-2.89 mm average surface errors. Thereby the level set segmentation (which is initialized by the learning-based segmentations) contributes with an 20%-40% increase in accuracy.

Lymph node detection and measurement is a difficult and important part of cancer treatment. In this paper we present a robust and effective learning-based method for the automatic detection of solid lymph nodes from Computed Tomography data. The contributions of the paper are the following. First, it presents a learning based approach to lymph node detection based on Marginal Space Learning. Second, it presents an efficient MRF-based segmentation method for solid lymph nodes. Third, it presents two new sets of features, one set self-aligning to the local gradients and another set based on the segmentation result. An extensive evaluation on 101 volumes containing 362 lymph nodes shows that this method obtains a 82.3% detection rate at 1 false positive per volume, with an average running time of 5-20 seconds per volume.

C-arm CT is an emerging imaging technique in transcatheter aortic valve implantation (TAVI) surgery. Automatic aorta segmentation and valve landmark detection in a C-arm CT volume has important applications in TAVI by providing valuable 3D measurements for surgery planning. Overlaying 3D segmentation onto 2D real time fluoroscopic images also provides critical visual guidance during the surgery. In this paper, we present a part-based aorta segmentation approach, which can handle aorta structure variation in case that the aortic arch and descending aorta are missing in the volume. The whole aorta model is split into four parts: aortic root, ascending aorta, aortic arch, and descending aorta. Discriminative learning is applied to train a detector for each part separately to exploit the rich domain knowledge embedded in an expert-annotated dataset. Eight important aortic valve landmarks (three aortic hinge points, three commissure points, and two coronary ostia) are also detected automatically in our system. Under the guidance of the detected landmarks, the physicians can deploy the prosthetic valve properly. Our approach is robust under variations of contrast agent. Taking about 1.4 seconds to process one volume, it is also computationally efficient.

Automatic segmentation of the esophagus from CT data is a challenging problem. Its wall consists of muscle tissue, which has low contrast in CT. Sometimes it is filled with air or remains of orally given contrast agent. While air holes are a clear hint to a human when searching for the esophagus, we found that they are rather distracting to discriminative models of the appearance because of their similarity to the trachea and to lung tissue. However, air inside the respiratory organs can be segmented easily. In this paper, we propose to combine a model based segmentation algorithm of the esophagus with a spatial probability map generated from detected air. Threefold cross-validation on 144 datasets showed that this probability map, combined with a technique that puts more focus on hard cases, increases accuracy by 22%. In contrast to prior work, our method is not only automatic on a manually selected region of interest, but on a whole thoracic CT scan, while our mean segmentation error of 1.80mm is even better.

Automated segmentation of the esophagus in CT images is of high value to radiologists for oncological examinations of the mediastinum. It can serve as a guideline and prevent confusion with pathological tissue. However, segmentation is a challenging problem due to low contrast and versatile appearance of the esophagus. In this paper, a two step method is proposed which first finds the approximate shape using a "detect and connect" approach. A classifier is trained to find short segments of the esophagus which are approximated by an elliptical model. Recently developed techniques in discriminative learning and pruning of the search space enable a rapid detection of possible esophagus segments. Prior shape knowledge of the complete esophagus is modeled using a Markov chain framework, which allows efficient inferrence of the approximate shape from the detected candidate segments. In a refinement step, the surface of the detected shape is non-rigidly deformed to better fit the organ boundaries. In contrast to previously proposed methods, no user interaction is required. It was evaluated on 117 datasets and achieves a mean segmentation error of 2.28mm with less than 9s computation time.

In this paper, we propose an algorithm for fast and automatic Doppler gate localization in spectral Doppler echocardiography using the B-mode image information. The algorithm has two components: 1) cardiac standard view classification and 2) gate location inference. For cardiac view classification, we incorporate the probabilistic boosting network (PBN) principle to local-structure-dependent object classification, which speeds up the processing time as it breaks down the computational dependency on the number of classes. The gate location is computed using a data-driven shape inference approach. Clinical evaluation was performed by implementing the algorithm on an ultrasound system. Experiment results show that the performance of the proposed algorithm is comparable to the Doppler gate placement by an expert user. To the best of our knowledge, this is the first algorithm that provides a real time solution to the automated Doppler gate placement in the clinical environment.

Doppler echocardiography is widely used for functional assessment of heart valves such as mitral valve. In current clinical work flow, to extract Doppler measurements, the envelopes of acquired Doppler spectra are manually traced. We propose a robust algorithm for automatically tracing the envelopes of mitral valve inflow Doppler spectra, which exhibit a large amount of variations in envelope shape and image appearance due to various disease conditions, patient/sonographer/instrument differences, etc. The algorithm is learning-based and capable of fully automatic detection and segmentation of the mitral inflow structures. Experiments show that the algorithm, running within one second, yields comparable performance to experts.

We propose a pairwise active appearance model (PAAM) to characterize statistical regularities in shape, appearance, and motion presented by a target that undergoes a series of motion phases, such as the left ventricle in echocardiography. The PAAM depicts the transition in motion phase through a Markov chain and the transition in both shape and appearance through a conditional Gaussian distribution. We learn from a database the joint Gaussian distribution of the shapes and appearances belonging to two consecutive motion phases (i.e., a pair of motion phases), from which we analytically compute the conditional Gaussian distribution. We utilize the PAAM in tracking the left ventricle contour in echocardiography and obtain improved tracking results in terms of localization accuracy when compared with expert-specified contours.