Search

Carotid ultrasound measurement of total plaque area (TPA) provides a method for quantifying carotid plaque burden and monitoring changes in carotid atherosclerosis in response to medical treatment. Plaque boundary segmentation is required to generate the TPA measurement; however, training of observers and manual delineation are time consuming. Thus, our objective was to develop an automated plaque segmentation method to generate TPA from longitudinal carotid ultrasound images. In this study, a deep learning-based method, modified U-Net, was used to train the segmentation model and generate TPA measurement. A total of 510 plaques from 144 patients were used in our study, where the Monte Carlo cross-validation was used by randomly splitting the data set into 2/3 and 1/3 for training and testing. Two observers were trained to manually delineate the 510 plaques separately, which were used as the ground-truth references. Two U-Net models (M1 and M2) were trained using the two different ground-truth data sets from the two observers to evaluate the accuracy, variability and sensitivity on the ground-truth data sets used for training our method. The results of the algorithm segmentations of the two models yielded strong agreement with the two manual segmentations with the Pearson correlation coefficient r = 0.989 (p < 0.0001) and r = 0.987 (p < 0.0001). Comparison of the U-Net and manual segmentations resulted in mean TPA differences of 0.05 ± 7.13 mm 2 (95% confidence interval: 14.02-13.02 mm 2 ) and 0.8 ± 8.7 mm 2 (17.85-16.25 mm 2 ) for the two models, which are small compared with the TPA range in our data set from 4.7 to 312.8 mm 2 . Furthermore, the mean time to segment a plaque was only 8.3 ± 3.1 ms. The presented deep learning-based method described has sufficient accuracy with a short computation time and exhibits high agreement between the algorithm and manual TPA measurements, suggesting that the method could be used to measure TPA and to monitor the progression and regression of carotid atherosclerosis., (Copyright © 2021 World Federation for Ultrasound in Medicine & Biology. Published by Elsevier Inc. All rights reserved.)

Measurement of total-plaque-area (TPA) is important for determining long term risk for stroke and monitoring carotid plaque progression. Since delineation of carotid plaques is required, a deep learning method can provide automatic plaque segmentations and TPA measurements; however, it requires large datasets and manual annotations for training with unknown performance on new datasets. A UNet++ ensemble algorithm was proposed to segment plaques from 2D carotid ultrasound images, trained on three small datasets (n = 33, 33, 34 subjects) and tested on 44 subjects from the SPARC dataset (n = 144, London, Canada). The ensemble was also trained on the entire SPARC dataset and tested with a different dataset (n = 497, Zhongnan Hospital, China). Algorithm and manual segmentations were compared using Dice-similarity-coefficient (DSC), and TPAs were compared using the difference ( ∆TPA), Pearson correlation coefficient (r) and Bland-Altman analyses. Segmentation variability was determined using the intra-class correlation coefficient (ICC) and coefficient-of-variation (CoV). For 44 SPARC subjects, algorithm DSC was 83.3-85.7%, and algorithm TPAs were strongly correlated (r = 0.985-0.988; p < 0.001) with manual results with marginal biases (0.73-6.75) mm 2 using the three training datasets. Algorithm ICC for TPAs (ICC = 0.996) was similar to intra- and inter-observer manual results (ICC = 0.977, 0.995). Algorithm CoV = 6.98% for plaque areas was smaller than the inter-observer manual CoV (7.54%). For the Zhongnan dataset, DSC was 88.6% algorithm and manual TPAs were strongly correlated (r = 0.972, p < 0.001) with ∆TPA = -0.44 ±4.05 mm 2 and ICC = 0.985. The proposed algorithm trained on small datasets and segmented a different dataset without retraining with accuracy and precision that may be useful clinically and for research.

Purpose: Quantification of carotid plaques has been shown to be important for assessing as well as monitoring the progression and regression of carotid atherosclerosis. Various metrics have been proposed and methods of measurements ranging from manual tracing to automated segmentations have also been investigated. Of those metrics, quantification of carotid plaques by measuring vessel-wall-volume (VWV) using the segmented media-adventitia (MAB) and lumen-intima (LIB) boundaries has been shown to be sensitive to temporal changes in carotid plaque burden. Thus, semi-automatic MAB and LIB segmentation methods are required to help generate VWV measurements with high accuracy and less user interaction., Methods: In this paper, we propose a semiautomatic segmentation method based on deep learning to segment the MAB and LIB from carotid three-dimensional ultrasound (3DUS) images. For the MAB segmentation, we convert the segmentation problem to a pixel-by-pixel classification problem. A dynamic convolutional neural network (Dynamic CNN) is proposed to classify the patches generated by sliding a window along the norm line of the initial contour where the CNN model is fine-tuned dynamically in each test task. The LIB is segmented by applying a region-of-interest of carotid images to a U-Net model, which allows the network to be trained end-to-end for pixel-wise classification., Results: A total of 144 3DUS images were used in this development, and a threefold cross-validation technique was used for evaluation of the proposed algorithm. The proposed algorithm-generated accuracy was significantly higher than the previous methods but with less user interactions. Comparing the algorithm segmentation results with manual segmentations by an expert showed that the average Dice similarity coefficients (DSC) were 96.46 ± 2.22% and 92.84 ± 4.46% for the MAB and LIB, respectively, while only an average of 34 s (vs 1.13, 2.8 and 4.4 min in previous methods) was required to segment a 3DUS image. The interobserver experiment indicated that the DSC was 96.14 ± 1.87% between algorithm-generated MAB contours of two observers' initialization., Conclusions: Our results showed that the proposed carotid plaque segmentation method obtains high accuracy and repeatability with less user interactions, suggesting that the method could be used in clinical practice to measure VWV and monitor the progression and regression of carotid plaques., (© 2019 The Authors Medical Physics published by Wiley Periodicals, Inc. on behalf of American Association of Physicists in Medicine.)

Background: Fetal brain magnetic resonance imaging (MRI)‐based gestational age prediction has been widely used to characterize normal fetal brain development and diagnose congenital brain malformations. Purpose: The uncertainty of fetal position and external interference leads to variable localization and direction of the fetal brain. In addition, pregnant women typically concentrate on receiving MRI scans during the fetal anomaly scanning week, leading to an imbalanced distribution of fetal brain MRI data. The above‐mentioned problems pose great challenges for deep learning‐based fetal brain MRI gestational age prediction. Methods: In this study, a pyramid squeeze attention (PSA)‐guided dynamic feature fusion CNN (PDFF‐CNN) is proposed to robustly predict gestational ages from fetal brain MRI images on an imbalanced dataset. PDFF‐CNN contains four components: transformation module, feature extraction module, dynamic feature fusion module, and balanced mean square error (MSE) loss. The transformation and feature extraction modules are employed by using the PSA to learn multiscale and multi‐orientation feature representations in a parallel weight‐sharing Siamese network. The dynamic feature fusion module automatically learns the weights of feature vectors generated in the feature extraction module to dynamically fuse multiscale and multi‐orientation brain sulci and gyri features. Considering the fact of the imbalanced dataset, the balanced MSE loss is used to mitigate the negative impact of imbalanced data distribution on gestational age prediction performance. Results: Evaluated on an imbalanced fetal brain MRI dataset of 1327 routine clinical T2‐weighted MRI images from 157 subjects, PDFF‐CNN achieved promising gestational age prediction performance with an overall mean absolute error of 0.848 weeks and an R2$R^2$ of 0.904. Furthermore, the attention activation maps of PDFF‐CNN were derived, which revealed regional features that contributed to gestational age prediction at each gestational stage. Conclusions: These results suggest that the proposed PDFF‐CNN might have broad clinical applicability in guiding treatment interventions and delivery planning, which has the potential to be helpful with prenatal diagnosis. [ABSTRACT FROM AUTHOR]

Triple-negative breast cancer (TNBC) is a relatively aggressive breast cancer subtype due to tumor relapse, drug resistance, and multi-organ metastatic properties. Identifying reliable biomarkers to predict prognosis and precisely guide TNBC immunotherapy is still an unmet clinical need. To address this issue, we successfully constructed a novel 25 machine learning (ML) algorithms-based immune infiltrating cell (IIC) associated signature of TNBC (MLIIC), achieved by multiple transcriptome data of purified immune cells, TNBC cell lines, and TNBC entities. The TSI index was employed to determine IIC-RNAs that were accompanied by an expression pattern of upregulation in immune cells and downregulation in TNBC cells. LassoLR, Boruta, Xgboost, SVM, RF, and Pamr were utilized for further obtaining the optimal IIC-RNAs. Following univariate Cox regression analysis, LassoCox, CoxBoost, and RSF were utilized for the dimensionality reduction of IIC-RNAs from a prognostic perspective. RSF, Ranger, ObliqueRSF, Rpart, CoxPH, SurvivalSVM, CoxBoost, GlmBoost, SuperPC, StepwiseCox, Enet, LassoCox, CForest, Akritas, BlackBoost, PlsRcox, SurvReg, GBM, and CTree were used for determining the most potent MLIIC signature. Consequently, this MLIIC signature was correlated significantly with survival status validated by four independent TNBC cohorts. Also, the MLIIC signature had a superior predictive capability for TNBC prognosis, compared with 148 previously reported signatures. In addition, MLIIC signature scores developed by immunofluorescent staining of tissue arrays from TNBC patients showed a substantial prognostic value. In TNBC immunotherapy, the low MLIIC profile demonstrated significant immune-responsive efficacy in a dataset of multiple cancer types. MLIIC signature could also predict m6A epigenetic regulation which controls T cell homeostasis. Therefore, this well-established MLIIC signature is a robust predictive indicator for TNBC prognosis and the benefit of immunotherapy, thus providing an efficient tool for combating TNBC. [ABSTRACT FROM AUTHOR]

Vessel wall volume (VWV) is a 3-D ultrasound measurement for the assessment of therapy in patients with carotid atherosclerosis. Deep learning can be used to segment the media–adventitia boundary (MAB) and lumen–intima boundary (LIB) and to quantify VWV automatically; however, it typically requires large training data sets with expert manual segmentation, which are difficult to obtain. In this study, a UNet++ ensemble approach was developed for automated VWV measurement, trained on five small data sets (n = 30 participants) and tested on 100 participants with clinically diagnosed coronary artery disease enrolled in a multicenter CAIN trial. The Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), Pearson correlation coefficient (r), Bland–Altman plots and coefficient of variation (CoV) were used to evaluate algorithm segmentation accuracy, agreement and reproducibility. The UNet++ ensemble yielded DSCs of 91.07%–91.56% and 87.53%–89.44% and ASSDs of 0.10–0.11 mm and 0.33–0.39 mm for the MAB and LIB, respectively; the algorithm VWV measurements were correlated (r = 0.763–0.795, p < 0.001) with manual segmentations, and the CoV for VWV was 8.89%. In addition, the UNet++ ensemble trained on 30 participants achieved a performance similar to that of U-Net and Voxel-FCN trained on 150 participants. These results suggest that our approach could provide accurate and reproducible carotid VWV measurements using relatively small training data sets, supporting deep learning applications for monitoring atherosclerosis progression in research and clinical trials. [ABSTRACT FROM AUTHOR]

The method of tactile perception can accurately reflect the contact state by collecting force and torque information, but it is not sensitive to the changes in position and posture between assembly objects. The method of visual perception is very sensitive to changes in pose and posture between assembled objects, but they cannot accurately reflect the contact state, especially since the objects are occluded from each other. The robot will perceive the environment more accurately if visual and tactile perception can be combined. Therefore, this paper proposes the alignment method of combined perception for the peg-in-hole assembly with self-supervised deep reinforcement learning. The agent first observes the environment through visual sensors and then predicts the action of the alignment adjustment based on the visual feature of the contact state. Subsequently, the agent judges the contact state based on the force and torque information collected by the force/torque sensor. And the action of the alignment adjustment is selected according to the contact state and used as a visual prediction label. Whereafter, the network of visual perception performs backpropagation to correct the network weights according to the visual prediction label. Finally, the agent will have learned the alignment skill of combined perception with the increase of iterative training. The robot system is built based on CoppeliaSim for simulation training and testing. The simulation results show that the method of combined perception has higher assembly efficiency than single perception. [ABSTRACT FROM AUTHOR]

The adjustment times of the attitude alignment are fluctuated due to the fluctuation of the contact force signal caused by the disturbing moments in the compliant peg-in-hole assembly. However, these fluctuations are difficult to accurately measure or definition as a result of many uncertain factors in the working environment. It is worth noting that gravitational disturbing moments and inertia moments significantly impact these fluctuations, in which the changes of the peg concerning the mass and the length have a crucial influence on them. In this paper, a visual grasping strategy based on deep reinforcement learning is proposed for peg-in-hole assembly. Firstly, the disturbing moments of assembly are analyzed to investigate the factors for the fluctuation of assembly time. Then, this research designs a visual grasping strategy, which establishes a mapping relationship between the grasping position and the assembly time to improve the assembly efficiency. Finally, a robotic system for the assembly was built in V-REP to verify the effectiveness of the proposed method, and the robot can complete the training independently without human intervention and manual labeling in the grasping training process. The simulated results show that this method can improve assembly efficiency by 13.83%. And, when the mass and the length of the peg change, the proposed method is still effective for the improvement of assembly efficiency. [ABSTRACT FROM AUTHOR]

Total Plaque Area (TPA) measurement is critical for early diagnosis and intervention of carotid atherosclerosis in individuals with high risk for stroke. The delineation of the carotid plaques is necessary for TPA measurement, and deep learning methods can automatically segment the plaque and measure TPA from carotid ultrasound images. A large number of labeled images is essential for training a good deep learning model, but it is very difficult to collect such large labeled datasets for carotid image segmentation in clinical practice. Self-supervised learning can provide a possible solution to improve the deep-learning models on small labeled training datasets by designing a pretext task to pre-train the models without using the segmentation masks. However, the existing self-supervised learning methods do not consider the feature presentations of object contours. In this paper, we propose an image registration-based self-supervised learning method and a stacked U-Net (SSL-SU-Net) for carotid plaque ultrasound image segmentation, which can better exploit the semantic features of carotid plaque contours in self-supervised task training. Our network was trained on different numbers of labeled images (n = 10, 33, 50 and 100 subjects) and tested on 44 subjects from the SPARC dataset (n = 144, London, Canada). The network trained on the entire SPARC dataset was then directly applied to an independent dataset collected in Zhongnan hospital (n = 497, Wuhan, China). For the 44 subjects tested on the SPARC dataset, our method yielded a DSC of 80.25-89.18% and the produced TPA measurements, which were strongly correlated with manual segmentation (r = 0.965-0.995, ρ < 0.0001). For the Zhongnan dataset, the DSC was 90.3% and algorithm TPAs were strongly correlated with manual TPAs (r = 0.985, ρ < 0.0001). The results demonstrate that our proposed method yielded excellent performance and good generalization ability when trained on a small labeled dataset, facilitating the use of deep learning in carotid ultrasound image analysis and clinical practice. The code of our algorithm is available https://github.com/a610lab/Registration-SSL. • A registration-based self-supervised learning algorithm was applied to carotid plaque segmentation on small labeled training datasets. • The proposed registration task makes the network focus on the segmentation target contour, thereby improving the segmentation performance. • A level-set and least-squares-based deformation method is proposed to generate appropriate image pairs for the image registration task. • A Stacked U-Net architecture is proposed to produce more refined segmentation results. [ABSTRACT FROM AUTHOR]

The rapid and precise segmentation of cell nuclei from hematoxylin and eosin-stained tissue images is an essential clinical undertaking with significant implications for various clinical applications. The segmentation of cell nuclei poses specific challenges due to the inherent instability of nuclear morphology and the complexity of the segmentation environments. Furthermore, previous studies have primarily relied on small-scale and limited-diverse datasets, potentially hindering their applicability to clinical tasks. This study introduces a novel approach, the Double-stage Codec Attention Network, designed to automatically and accurately segment nuclei. Specifically, we present a hierarchical feature extraction module, which maximizes the utilization of cell nuclei's morphological characteristics in the tissue, thereby providing critical semantic information for nucleus segmentation. Furthermore, the feature selection units are employed to enhance relevant features and suppress interfering ones, thereby enhancing the overall expressive capacity of the information. The multi-scale deep feature fusion module utilizes interrelated encoder–decoder connections to jointly optimize and integrate this information, generating a robust hierarchical feature pyramid. Finally, the feature attention fusion mechanism captures spatial and directional information, aiding the model in the accurate localization and recognition of cell nuclei. We rigorously evaluated our proposed method using the PanNuke dataset, the largest comprehensive histology dataset of cancer tissues. In terms of the average F1-score across all segmentation classes in the PanNuke dataset, DSCA-Net outperforms other state-of-the-art models such as DeepLabV3+, TransUNet, Triple U-net, and TransNuSeg by 1.38, 1.44, 2.64, and 1.02, respectively. Additionally, DSCA-Net shows excellent efficiency in generating predictive images, outperforming all comparative models. • We design a new feature extraction module, HFE. • We develop an inverted U-shaped module, DSFF, that can handle depth features. • We construct a new skip-join structure, FAM. • DSCA-Net has good robustness and generalizability. [ABSTRACT FROM AUTHOR]

Carotid atherosclerotic plaques cause stroke when plaques rupture and clog the blood vessels that deliver blood to brain. Ultrasound measurements (i.e. total-plaque-area and intima-media-thickness) are mainly used to monitor the progression and regression of plaques. Recently, deep learning has provided powerful tools for ultrasound carotid image segmentation. However, the performances of deep learning models vary on different network architectures. In this paper, we report on the development of an adaptively weighted ensemble of multiple convolutional neural networks (CNNs) for carotid ultrasound image segmentation, aiming at combining the advantages of different CNN models to achieve higher accuracy and better generalization performance. During the joint training of the ensemble networks, the model weights and sample weights were combined to improve the segmentation performance. This adaptively weighted ensemble algorithm was applied to three UNet++ models with different backbones (ResNet152, DenseNet169 and VGG19), and evaluated on 510 carotid ultrasound images from 144 subjects who were followed in the Stroke Prevention and Atherosclerosis Research Centre (SPARC, London, Canada). The experimental results show that our method increases the segmentation accuracy, and reduces the distance errors as compared to using a single classifier, three ensemble algorithms (average weighting, majority voting and SegNet-UNet+) and a published carotid segmentation algorithm. With high accuracy and low variance, the proposed adaptively weighted ensemble model could be used to measure carotid plaques in clinical practice and clinical trials. • An adaptively weighted ensemble algorithm is developed by combining the advantages of different CNN models. • The model and sample weighting strategies are proposed to improve the segmentation performance in the ensemble algorithm training. • This ensemble algorithm is useful for carotid plaque measurement in clinical practice and trials. [ABSTRACT FROM AUTHOR]

New ongoing rural construction has resulted in an extensive mixture of new settlements with old ones in the rural areas of China. Understanding the spatial characteristic of these rural settlements is of crucial importance as it provides essential information for land management and decision-making. Despite a great advance in High Spatial Resolution (HSR) satellite images and deep learning techniques, it remains a challenging task for mapping rural settlements accurately because of their irregular morphology and distribution pattern. In this study, we proposed a novel framework to map rural settlements by leveraging the merits of Gaofen-2 HSR images and representation learning of deep learning. We combined a dilated residual convolutional network (Dilated-ResNet) and a multi-scale context subnetwork into an end-to-end architecture in order to learn high resolution feature representations from HSR images and to aggregate and refine the multi-scale features extracted by the aforementioned network. Our experiment in Tongxiang city showed that the proposed framework effectively mapped and discriminated rural settlements with an overall accuracy of 98% and Kappa coefficient of 85%, achieving comparable and improved performance compared to other existing methods. Our results bring tangible benefits to support other convolutional neural network (CNN)-based methods in accurate and timely rural settlement mapping, particularly when up-to-date ground truth is absent. The proposed method does not only offer an effective way to extract rural settlement from HSR images but open a new opportunity to obtain spatial-explicit understanding of rural settlements. [ABSTRACT FROM AUTHOR]