Search

Background: Deep learning has presented great potential in accurate MR image segmentation when enough labeled data are provided for network optimization. However, manually annotating three‐dimensional (3D) MR images is tedious and time‐consuming, requiring experts with rich domain knowledge and experience. Purpose: To build a deep learning method exploring sparse annotations, namely only a single two‐dimensional slice label for each 3D training MR image. Study Type: Retrospective. Population: Three‐dimensional MR images of 150 subjects from two publicly available datasets were included. Among them, 50 (1377 image slices) are for prostate segmentation. The other 100 (8800 image slices) are for left atrium segmentation. Five‐fold cross‐validation experiments were carried out utilizing the first dataset. For the second dataset, 80 subjects were used for training and 20 were used for testing. Field Strength/Sequence: 1.5 T and 3.0 T; axial T2‐weighted and late gadolinium‐enhanced, 3D respiratory navigated, inversion recovery prepared gradient echo pulse sequence. Assessment: A collaborative learning method by integrating the strengths of semi‐supervised and self‐supervised learning schemes was developed. The method was trained using labeled central slices and unlabeled noncentral slices. Segmentation performance on testing set was reported quantitatively and qualitatively. Statistical Tests: Quantitative evaluation metrics including boundary intersection‐over‐union (B‐IoU), Dice similarity coefficient, average symmetric surface distance, and relative absolute volume difference were calculated. Paired t test was performed, and P < 0.05 was considered statistically significant. Results: Compared to fully supervised training with only the labeled central slice, mean teacher, uncertainty‐aware mean teacher, deep co‐training, interpolation consistency training (ICT), and ambiguity‐consensus mean teacher, the proposed method achieved a substantial improvement in segmentation accuracy, increasing the mean B‐IoU significantly by more than 10.0% for prostate segmentation (proposed method B‐IoU: 70.3% ± 7.6% vs. ICT B‐IoU: 60.3% ± 11.2%) and by more than 6.0% for left atrium segmentation (proposed method B‐IoU: 66.1% ± 6.8% vs. ICT B‐IoU: 60.1% ± 7.1%). Data Conclusions: A collaborative learning method trained using sparse annotations can segment prostate and left atrium with high accuracy. Level of Evidence: 0 Technical Efficacy: Stage 1 [ABSTRACT FROM AUTHOR]

Objective. Training neural networks for pixel-wise or voxel-wise image segmentation is a challenging task that requires a considerable amount of training samples with highly accurate and densely delineated ground truth maps. This challenge becomes especially prominent in the medical imaging domain, where obtaining reliable annotations for training samples is a difficult, time-consuming, and expert-dependent process. Therefore, developing models that can perform well under the conditions of limited annotated training data is desirable. Approach. In this study, we propose an innovative framework called the extremely sparse annotation neural network (ESA-Net) that learns with only the single central slice label for 3D volumetric segmentation which explores both intra-slice pixel dependencies and inter-slice image correlations with uncertainty estimation. Specifically, ESA-Net consists of four specially designed distinct components: (1) an intra-slice pixel dependency-guided pseudo-label generation module that exploits uncertainty in network predictions while generating pseudo-labels for unlabeled slices with temporal ensembling; (2) an inter-slice image correlation-constrained pseudo-label propagation module which propagates labels from the labeled central slice to unlabeled slices by self-supervised registration with rotation ensembling; (3) a pseudo-label fusion module that fuses the two sets of generated pseudo-labels with voxel-wise uncertainty guidance; and (4) a final segmentation network optimization module to make final predictions with scoring-based label quantification. Main results. Extensive experimental validations have been performed on two popular yet challenging magnetic resonance image segmentation tasks and compared to five state-of-the-art methods. Significance. Results demonstrate that our proposed ESA-Net can consistently achieve better segmentation performances even under the extremely sparse annotation setting, highlighting its effectiveness in exploiting information from unlabeled data. [ABSTRACT FROM AUTHOR]

Traditional supervised learning requires ground truth labels for training, whose collection however is difficult in many cases. Recently, crowdsourcing has established itself as an efficient labeling solution by resorting to non-expert crowds. To reduce the labeling error effects, one common practice is to distribute each instance to multiple workers, whereas each worker only annotates a subset of data, resulting in the sparse annotation phenomenon. In this paper, we show that when meeting with class-imbalance, i.e., even when the groundtruth labels are slightly imbalanced, the sparse annotations are prone to be skewly distributed and would bias the learning algorithm severely. To combat this issue, we propose one Distribution Aware Self-training based Crowdsourcing learning (DASC) approach, which supplements the sparse annotations by adding confident pseudo-annotations and at the same time re-balancing the annotation distribution. Specifically, we propose one distribution aware confidence measure to select the most confident pseudo-annotations, with minority/majority classes selected more/less frequently. As a universal framework, DASC is applicable to various crowdsourcing methods for consistent performance gains. We conduct extensive experiments over real-world crowdsourcing benchmarks, from slight to heavy imbalance ratio, with various annotation sparsity levels, and show that DASC substantially improves previous crowdsourcing models by 2 % - 20 % absolute test accuracy, and yields much more balanced annotations. [ABSTRACT FROM AUTHOR]

Across industries, creating new datasets often becomes necessary, e.g., for specialized machine learning tasks where no datasets are readily available. However, creating a dataset can be very complex and time-consuming. Especially for semantic segmentation tasks, when every pixel needs to be manually labeled. To quickly arrive at an initial model, it is desirable to drop most of the labeling requirements, e.g., by allowing sparse (lazy) annotations like scribbles. Sometimes these initial models already produce satisfactory results, in other cases, they could help to refine further and extend the dataset. However, segmentation models typically do not accept sparse labels for training. We propose a simple dropout procedure that allows a model to handle scribble and point annotations. The procedure focuses on being easily transferable to existing semantic segmentation implementations and methods. As a representative example, a chicken semantic segmentation dataset was employed for experiments. Although the model is trained on sparse annotation masks, it predicts dense masks. The result is a standalone model that does not require further human assistance after training. Two models trained on point annotations and scribbles were compared to a baseline model trained on dense masks. Although the baseline model performs best, our method can achieve comparable results with just one percent of labeled pixels. This holds as long as the sparse labels have good coverage. [ABSTRACT FROM AUTHOR]

This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 11% reaching an average user's accuracy of 88.63% and an average producer's accuracy of 88.59%, achieving state-of-art performance for tree species classification in tropical forests. [Display omitted] [ABSTRACT FROM AUTHOR]

4D CT imaging has a great potential for use in stroke workup. A fully convolutional neural network (CNN) for 3D multiclass segmentation in 4D CT is presented, which can be trained end-to-end from sparse 2D annotations. The CNN was trained and validated on 42 4D CT acquisitions of the brain of patients with suspicion of acute ischemic stroke. White matter, gray matter, cerebrospinal fluid, and vessels were annotated by two trained observers. The mean Dice coefficients, contour mean distances, and absolute volume differences were, respectively, 0.87 ± 0.04, 0.52 ± 0.47 mm, and 11.78 ± 9.55 % on a separate test set of five patients, which were similar to the average interobserver variability scores of 0.88 ± 0.03, 0.72 ± 0.93 mm, and 8.86 ± 7.65 % outperforming the current state of the art. The proposed method is, therefore, a promising deep neural network for multiclass segmentation in 4D spatiotemporal imaging data.

• Medical image segmentation typically faces limited datasets. • Dataset limitations are broadly grouped into scarce and weak annotations. • Scarce annotations can be addressed proactively via cost-effective annotation or by leveraging external labeled or unlabeled datasets. • Scarce annotations can be addressed reactively via various forms of conditional random fields as post processing. • Weak annotations may manifest as sparse, noisy, or only image-level labels. • Sparse and noisy annotations can leveraged via selective and noise-resilient loss functions, respectively. • Image-level labels can leveraged via various forms of class activation maps. • Recommended solutions based on a cost-gain trade-off are provided. The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations. In this article, we provide a detailed review of the solutions above, summarizing both the technical novelties and empirical results. We further compare the benefits and requirements of the surveyed methodologies and provide our recommended solutions. We hope this survey article increases the community awareness of the techniques that are available to handle imperfect medical image segmentation datasets. [ABSTRACT FROM AUTHOR]