Search

Spatial transcriptomics reveals the spatial distribution of genes in complex tissues, providing crucial insights into biological processes, disease mechanisms, and drug development. The prediction of gene expression based on cost-effective histology images is a promising yet challenging field of research. Existing methods for gene prediction from histology images exhibit two major limitations. First, they ignore the intricate relationship between cell morphological information and gene expression. Second, these methods do not fully utilize the different latent stages of features extracted from the images. To address these limitations, we propose a novel hypergraph neural network model, HGGEP, to predict gene expressions from histology images. HGGEP includes a gradient enhancement module to enhance the model's perception of cell morphological information. A lightweight backbone network extracts multiple latent stage features from the image, followed by attention mechanisms to refine the representation of features at each latent stage and capture their relations with nearby features. To explore higher-order associations among multiple latent stage features, we stack them and feed into the hypergraph to establish associations among features at different scales. Experimental results on multiple datasets from disease samples including cancers and tumor disease, demonstrate the superior performance of our HGGEP model than existing methods. [ABSTRACT FROM AUTHOR]

Accurate nuclei segmentation is essential for extracting quantitative information from histology images to support disease diagnosis and treatment decisions. However, precise segmentation is challenging due to the presence of clustered nuclei, varied morphologies, and the need to capture global spatial correlations. While state-of-the-art Transformer-based models employ tri-decoder architectures to decouple the segmentation task into nuclei, edges, and cluster edges segmentation, their complexity and long inference times hinder clinical integration. To address this, we introduce MoE-NuSeg, a novel Mixture of Experts (MoE) network that consolidates the tri-decoder into a single decoder. MoE-NuSeg employs three specialized experts for nuclei segmentation, edge delineation, and cluster edge detection, thereby mirroring the functionality of tri-decoders while surpassing their performance and reducing parameters by sharing attention heads. We propose a two-stage training strategy: the first stage independently trains the three experts, and the second stage fine-tunes their interactions to dynamically allocate the contributions of each expert using a learnable attention-based gating network. Evaluations across three datasets demonstrate that MoE-NuSeg outperforms the state-of-the-art methods, achieving an average increase of 0.99% in Dice coefficient, 1.14% in IoU and 0.92% in F1 Score, while reducing parameters by 30.1% and FLOPs by 40.2%. The code is available at https://github.com/deep-geo/MoE-NuSeg.

This paper presents a technique that can potentially help to determine the receptivity stage of the endometrium from histology images by automatically measuring the stromal nuclear changes. The presented technique is composed of an image segmentation model and the statistical evolution of segmented areas in hematoxylin and eosin (HE)-stained histology images. Three different endometrium receptivity stages, namely pre-receptive, post-receptive, and receptive, were compared. An ensemble-based AI model was proposed for histology image segmentation, which is based on individual UNet++, UNet, and ResNet34-UNet segmentation models. The performance of the ensemble-based technique was assessed using the Dice score and intersection over unit (IoU) values. In comparison to alternative segmentation architectures that were applied singly, the current ensemble-based method obtained higher Dice score (0.95) and IoU (0.90) values. The statistical comparison highlighted a noticeable difference in the number of nuclei and the size of the stroma tissue. The proposed technique demonstrated the positive potential for practical implementation for automatic endometrial tissue analysis. [ABSTRACT FROM AUTHOR]

Abstract Accurate segmentation of nuclei in histology images is essential for digital pathology. However, previous work should have considered integrating nuclei contour information into network learning. The main problems are (1) nuclei contour information needs to be fully considered, resulting in inaccurate spatial location of nuclei. (2) Small nuclei in dense regions or irregularly shaped nuclei cannot be segmented. (3) Mainstream networks consider more long-distance semantic information and larger receptive fields and do not consider the fusion of feature maps with different semantics. To solve the above problems, we have proposed a contour-based dual-path instance segmentation network. Firstly we reconstructed the nuclei contour information using algorithms and morphological manipulations. Provide input for subsequent contouring networks. Then we designed a dual-path network. It can extract nuclei features and contour features independently in the encoding stage and fuse the feature maps at different scales in the decoding stage. In the decoding stage, we use the attention module with a newly designed fusion mechanism, which allows us to fuse different semantics of feature maps for simple and efficient fusion. Finally, using the watershed algorithm, we use the network segmentation results to get the instance segmentation results of the nuclei. We used four publicly available datasets, including Kumar, CPM-15, CPM-17, and TNBC, which contain the nuclei of many organs and different forms of nuclei in the human body. Compared with the mainstream methods, we obtained the best AJI metrics, which are 0.656,0.717, and 0.602. In conclocess can effectively improve the performance of the nuclei instance segmentation network. It can accurately locate the scattered nuclei in space with small segmented nuclei in dense regions.

This paper presents a technique that can potentially help to determine the receptivity stage of the endometrium from histology images by automatically measuring the stromal nuclear changes. The presented technique is composed of an image segmentation model and the statistical evolution of segmented areas in hematoxylin and eosin (HE)-stained histology images. Three different endometrium receptivity stages, namely pre-receptive, post-receptive, and receptive, were compared. An ensemble-based AI model was proposed for histology image segmentation, which is based on individual UNet++, UNet, and ResNet34-UNet segmentation models. The performance of the ensemble-based technique was assessed using the Dice score and intersection over unit (IoU) values. In comparison to alternative segmentation architectures that were applied singly, the current ensemble-based method obtained higher Dice score (0.95) and IoU (0.90) values. The statistical comparison highlighted a noticeable difference in the number of nuclei and the size of the stroma tissue. The proposed technique demonstrated the positive potential for practical implementation for automatic endometrial tissue analysis.

Accurate segmentation of nuclei in histology images is essential for digital pathology. However, previous work should have considered integrating nuclei contour information into network learning. The main problems are (1) nuclei contour information needs to be fully considered, resulting in inaccurate spatial location of nuclei. (2) Small nuclei in dense regions or irregularly shaped nuclei cannot be segmented. (3) Mainstream networks consider more long-distance semantic information and larger receptive fields and do not consider the fusion of feature maps with different semantics. To solve the above problems, we have proposed a contour-based dual-path instance segmentation network. Firstly we reconstructed the nuclei contour information using algorithms and morphological manipulations. Provide input for subsequent contouring networks. Then we designed a dual-path network. It can extract nuclei features and contour features independently in the encoding stage and fuse the feature maps at different scales in the decoding stage. In the decoding stage, we use the attention module with a newly designed fusion mechanism, which allows us to fuse different semantics of feature maps for simple and efficient fusion. Finally, using the watershed algorithm, we use the network segmentation results to get the instance segmentation results of the nuclei. We used four publicly available datasets, including Kumar, CPM-15, CPM-17, and TNBC, which contain the nuclei of many organs and different forms of nuclei in the human body. Compared with the mainstream methods, we obtained the best AJI metrics, which are 0.656,0.717, and 0.602. In conclocess can effectively improve the performance of the nuclei instance segmentation network. It can accurately locate the scattered nuclei in space with small segmented nuclei in dense regions. [ABSTRACT FROM AUTHOR]

The paper proposes an integrated approach to the automated diagnosis of cervical intraepithelial neoplasia (CIN) in epithelial patches extracted from digital histology images. Experiments were conducted to determine the most suitable deep learning model for the dataset and fuse patch predictions to decide the final CIN grade of the histology samples. Seven candidate CNN architectures were assessed in this study. Three fusion methods were applied to the best CNN classifier. The model ensemble, combined CNN classifier and highest performing fusion method achieved an accuracy of 94.57%. This result shows significant improvement over the state-of-the-art classifiers for cervical cancer histopathology images. It is hoped that this work will contribute towards further research to automate diagnosis of CIN from digital histopathology images. [ABSTRACT FROM AUTHOR]

Histological examination of collagen fiber organization is essential for pathologists to observe the wound healing process. A convolutional neural network (CNN) can be utilized to visually analyze collagen fibers during tissue remodeling in histology images. In this study, a universal CNN (UCNN) independent of the histological staining process is proposed to classify the histology images of burn-induced scar tissues and characterize collagen fiber organization. Normal and scar tissues obtained from an in vivo rodent model are stained using Masson’s Trichrome (MT) and Hematoxylin & Eosin (H&E). The proposed universal model is trained using both MT- and H&E-stained histological image datasets over multiple scales with color augmentation, and classification accuracies of up to 98% and 97% are achieved for the MT- and H&E-stained image datasets, respectively. Regardless of the histological staining process used, the collagen characteristics are visualized by determining the density and directional variance of the normal and scar tissues by using the features extracted with the proposed universal model. Statistical analysis results demonstrated clear differences between scar and normal tissues in terms of collagen fiber organization. The proposed UCNN model can contribute to the development of an intelligent and efficient method that pathologists can use to rapidly evaluate wound healing and tissue remodeling.

The rapid development of spatial transcriptomics allows the measurement of RNA abundance at a high spatial resolution, making it possible to simultaneously profile gene expression, spatial locations of cells or spots, and the corresponding hematoxylin and eosin-stained histology images. It turns promising to predict gene expression from histology images that are relatively easy and cheap to obtain. For this purpose, several methods are devised, but they have not fully captured the internal relations of the 2D vision features or spatial dependency between spots. Here, we developed Hist2ST, a deep learning-based model to predict RNA-seq expression from histology images. Around each sequenced spot, the corresponding histology image is cropped into an image patch and fed into a convolutional module to extract 2D vision features. Meanwhile, the spatial relations with the whole image and neighbored patches are captured through Transformer and graph neural network modules, respectively. These learned features are then used to predict the gene expression by following the zero-inflated negative binomial distribution. To alleviate the impact by the small spatial transcriptomics data, a self-distillation mechanism is employed for efficient learning of the model. By comprehensive tests on cancer and normal datasets, Hist2ST was shown to outperform existing methods in terms of both gene expression prediction and spatial region identification. Further pathway analyses indicated that our model could reserve biological information. Thus, Hist2ST enables generating spatial transcriptomics data from histology images for elucidating molecular signatures of tissues. [ABSTRACT FROM AUTHOR]

With the progress of clinical imaging innovation and machine learning, the computer-assisted diagnosis of breast histology images has attracted broad attention. Nonetheless, the use of computer-assisted diagnoses has been blocked due to the incomprehensibility of customary classification models. In view of this question, we propose a novel method for Learning Binary Semantic Embedding (LBSE). In this study, bit balance and uncorrelation constraints, double supervision, discrete optimization and asymmetric pairwise similarity are seamlessly integrated for learning binary semantic-preserving embedding. Moreover, a fusion-based strategy is carefully designed to handle the intractable problem of parameter setting, saving huge amounts of time for boundary tuning. Based on the above-mentioned proficient and effective embedding, classification and retrieval are simultaneously performed to give interpretable image-based deduction and model helped conclusions for breast histology images. Extensive experiments are conducted on three benchmark datasets to approve the predominance of LBSE in different situations. [ABSTRACT FROM AUTHOR]

The evaluation of the morphology and organization of collagen fibers is critical in understanding wound healing and tissue remodeling after a thermal injury of the skin. However, histological analysis conducted by pathologists is often labor-intensive and limited to qualitative evaluations and scoring within a narrow field of view. In this study, we propose a convolutional neural network (CNN) model to classify Masson’s trichrome (MT)-stained histology images of burn-induced scar tissue and to characterize the microstructures of normal tissue and scar tissue in a quantitative manner. The scar tissue is created on in vivo rodent models and prepared for MT-stained histology slides after wound healing. A CNN model is developed, trained, and tested with various sizes of the histology images for classification and characterization. The proposed model classifies both normal tissue (i.e., without burn, as the control) and scar tissue at various scales with over 97% accuracy. The features acquired from the proposed CNN model visually characterizes the density and directional variance of the collagen fibers distributed in the dermal layers from whole histology images. The proposed deep learning technique can provide an objective and reliable method to rapidly assess and quantify wound repair and remodeling.

Artificial intelligence has enabled the automated diagnosis of several cancer types. We aimed to develop and validate deep learning models that automatically classify cervical intraepithelial neoplasia (CIN) based on histological images. Microscopic images of CIN3, CIN2, CIN1, and non-neoplasm were obtained. The performances of two pre-trained convolutional neural network (CNN) models adopting DenseNet-161 and EfficientNet-B7 architectures were evaluated and compared with those of pathologists. The dataset comprised 1106 images from 588 patients; images of 10% of patients were included in the test dataset. The mean accuracies for the four-class classification were 88.5% (95% confidence interval [CI], 86.3–90.6%) by DenseNet-161 and 89.5% (95% CI, 83.3–95.7%) by EfficientNet-B7, which were similar to human performance (93.2% and 89.7%). The mean per-class area under the receiver operating characteristic curve values by EfficientNet-B7 were 0.996, 0.990, 0.971, and 0.956 in the non-neoplasm, CIN3, CIN1, and CIN2 groups, respectively. The class activation map detected the diagnostic area for CIN lesions. In the three-class classification of CIN2 and CIN3 as one group, the mean accuracies of DenseNet-161 and EfficientNet-B7 increased to 91.4% (95% CI, 88.8–94.0%), and 92.6% (95% CI, 90.4–94.9%), respectively. CNN-based deep learning is a promising tool for diagnosing CIN lesions on digital histological images.

Colorectal cancer (CRC) is a major global health concern. Its early diagnosis is extremely important, as it determines treatment options and strongly influences the length of survival. Histologic diagnosis can be made by pathologists based on images of tissues obtained from a colonoscopic biopsy. Convolutional neural networks (CNNs)—i.e., deep neural networks (DNNs) specifically adapted to image data—have been employed to effectively classify or locate tumors in many types of cancer. Colorectal histology images of 28 normal and 29 tumor samples were obtained from the National Cancer Center, South Korea, and cropped into 6806 normal and 3474 tumor images. We developed five modifications of the system from the Visual Geometry Group (VGG), the winning entry in the classification task in the 2014 ImageNet Large Scale Visual Recognition Competition (ILSVRC) and examined them in two experiments. In the first experiment, we determined the best modified VGG configuration for our partial dataset, resulting in accuracies of 82.50%, 87.50%, 87.50%, 91.40%, and 94.30%, respectively. In the second experiment, the best modified VGG configuration was applied to evaluate the performance of the CNN model. Subsequently, using the entire dataset on the modified VGG-E configuration, the highest results for accuracy, loss, sensitivity, and specificity, respectively, were 93.48%, 0.4385, 95.10%, and 92.76%, which equates to correctly classifying 294 normal images out of 309 and 667 tumor images out of 719. [ABSTRACT FROM AUTHOR]

Evaluating a segmentation model is critical for constructing computer-aided diagnosis (CAD) systems, in which the segmented regions will be used for downstream analysis. However, existing segmentation evaluation metrics may not reflect every aspect of a trained segmentation model performance in the context of the whole slide tissue histology images (WSIs). Specifically, existing segmentation metrics generally ignore the impact of densely packed false positive pixels (DFP) in the WSI segmentation result. This study presents a new and efficient metric, named MAFaR, accounting for DFP regions in digital WSI segmentation. The proposed metric consists of two modules: 1) Estimation of DFP regions; 2) Calculation of MAFaR score. In module 1, a Gaussian Kernel Density Estimation method was used to estimate the density of the false positive (FP) regions in segmentation result (SR). Then a two-step Mean-shift clustering algorithm was applied to the high-density FP regions to estimate the DFP regions. In module 2, the ratio of DFP regions area to the positive regions area were used for MAFaR calculation. We conducted two experiments to evaluate the effectiveness of the MAFaR score. In the first experiment, we compared MAFaR with existing metrics related to FP regions, the proposed MAFaR score can reflect the impact of DFP regions. In the second experiment, MAFaR scores were compared with the manual evaluation scores given by three experienced engineers and found high correlation (Spearman's rank correlation coefficients greater than 0.7) and high agreement (Kendall coefficient = 0.839). Therefore, the MAFaR can be used with other segmentation metrics for evaluating WSI segmentation. [ABSTRACT FROM AUTHOR]

The study proposes an integrated approach to automated cervical intraepithelial neoplasia (CIN) diagnosis in epithelial patches extracted from digital histology images. The model ensemble, combined CNN classifier, and highest-performing fusion approach achieved an accuracy of 94.57%. This result demonstrates significant improvement over the state-of-the-art classifiers for cervical cancer histopathology images and promises further improvement in the automated diagnosis of CIN. [ABSTRACT FROM AUTHOR]

Objective: A new image instance segmentation method is proposed to segment individual glands (instances) in colon histology images. This process is challenging since the glands not only need to be segmented from a complex background, they must also be individually identified. Methods: We leverage the idea of image-to-image prediction in recent deep learning by designing an algorithm that automatically exploits and fuses complex multichannel information—regional, location, and boundary cues—in gland histology images. Our proposed algorithm, a deep multichannel framework, alleviates heavy feature design due to the use of convolutional neural networks and is able to meet multifarious requirements by altering channels. Results: Compared with methods reported in the 2015 MICCAI Gland Segmentation Challenge and other currently prevalent instance segmentation methods, we observe state-of-the-art results based on the evaluation metrics. Conclusion: The proposed deep multichannel algorithm is an effective method for gland instance segmentation. Significance: The generalization ability of our model not only enable the algorithm to solve gland instance segmentation problems, but the channel is also alternative that can be replaced for a specific task. [ABSTRACT FROM PUBLISHER]

Accurate classification of nuclei in histology images is essential for clinical diagnosis, prognosis, and therapeutic response prediction of cancer. However, this is still a challenging task due to (1) nuclei exhibiting a high level of heterogeneity within different types and (2) large intra-class variability including complex morphology and large variations of scale. To solve these problems, we propose a novel scale and region-enhanced decoding network based on the traditional U-shape structure for nuclei classification. We employ a nuclei detection head as region enhancement module in the decoding branch, which can enhance the nuclear regional information by locating the approximate bounding regions and provide more distinguish information for producing better feature maps of subsequent classification. Then, we propose a scale-aware feature fusion module, which fuses stage-wise feature maps generated from the decoder branch, to effectively learn multi-scale features. Finally, we utilize a scale attention module to calibrate the features and adapt to the most suitable scale in the hybrid multi-scale feature maps. In comparison with several state-of-the-art methods on two publicly available colonic cancer nuclei classification datasets, namely ConSep and Lizard, the proposed method obtains the highest accuracy of 0.860 and 0.927, respectively. It also achieves the highest accuracy of 0.838 on the PanNuke dataset collected from different tissues at different magnitudes. The independent validation on two subsets of the Lizard dataset indicates the proposed method obtains the highest accuracy. In conclusion, the proposed method can greatly improve classification performance, particularly for challenging nuclei with complex contexts and large-scale variations. • A SFF module is designed to effectively fuse multi-scale context information in different stages. • A SA module is proposed to automatically learn image-specific weight for calibrating the features. • Compared to the state-of-the-art methods, the proposed method achieves improved performances of nuclei classification. [ABSTRACT FROM AUTHOR]

Artificial intelligence has enabled the automated diagnosis of several cancer types. We aimed to develop and validate deep learning models that automatically classify cervical intraepithelial neoplasia (CIN) based on histological images. Microscopic images of CIN3, CIN2, CIN1, and non-neoplasm were obtained. The performances of two pre-trained convolutional neural network (CNN) models adopting DenseNet-161 and EfficientNet-B7 architectures were evaluated and compared with those of pathologists. The dataset comprised 1106 images from 588 patients; images of 10% of patients were included in the test dataset. The mean accuracies for the four-class classification were 88.5% (95% confidence interval [CI], 86.3–90.6%) by DenseNet-161 and 89.5% (95% CI, 83.3–95.7%) by EfficientNet-B7, which were similar to human performance (93.2% and 89.7%). The mean per-class area under the receiver operating characteristic curve values by EfficientNet-B7 were 0.996, 0.990, 0.971, and 0.956 in the non-neoplasm, CIN3, CIN1, and CIN2 groups, respectively. The class activation map detected the diagnostic area for CIN lesions. In the three-class classification of CIN2 and CIN3 as one group, the mean accuracies of DenseNet-161 and EfficientNet-B7 increased to 91.4% (95% CI, 88.8–94.0%), and 92.6% (95% CI, 90.4–94.9%), respectively. CNN-based deep learning is a promising tool for diagnosing CIN lesions on digital histological images. [ABSTRACT FROM AUTHOR]