Search

Microscopic image analysis technology helps solve the inadvertences of artificial traditional methods in disease, wastewater treatment, and environmental change monitoring analysis. Convolutional neural network (CNN) play an important role in microscopic image analysis. Image segmentation, in which U-Net is increasingly applied in microscopic image segmentation, is a crucial step in detection, tracking, monitoring, feature extraction, modelling, and analysis. This paper comprehensively reviews the development history of U-Net, analyses several research results of various segmentation methods since the emergence of U-Net, and conducts a comprehensive review of related papers. This paper summarised the improved methods of U-Net and then listed the existing significance of image segmentation techniques and their improvements introduced over the years. Finally, focusing on the different improvement strategies of U-Net in different papers, the related work of each application target is reviewed according to detailed technical categories to facilitate future research. Researchers can see the dynamics of the transmission of technological development and keep up with future trends in this interdisciplinary field. [ABSTRACT FROM AUTHOR]

Currently, numerous high-precision models have been proposed for semantic segmentation, but the model parameters are large and the segmentation speed is slow. Real-time semantic segmentation for urban scenes necessitates a balance between accuracy, inference speed, and model size. In this paper, we present an efficient solution to this challenge, efficient asymmetric attention module net (EAAMNet) for the semantic segmentation of urban scenes, which adopts an asymmetric encoder–decoder structure. The encoder part of the network utilizes an efficient asymmetric attention module to form the network backbone. In the decoding part, we propose a lightweight multi-feature fusion decoder that can maintain good segmentation accuracy with a small number of parameters. Our extensive evaluations demonstrate that EAAMNet achieves a favorable equilibrium between segmentation efficiency, model parameters, and segmentation accuracy, rendering it highly suitable for real-time semantic segmentation in urban scenes. Remarkably, EAAMNet attains a 73.31% mIoU at 128 fps on Cityscapes and a 69.32% mIoU at 141 fps on CamVid without any pre-training. Compared to state-of-the-art models, our approach not only matches their model parameters but also enhances accuracy and increases speed. [ABSTRACT FROM AUTHOR]

The segmentation of neuronal morphology in electron microscopy images is crucial for the analysis and understanding of neuronal function. However, most of the existing segmentation methods are not suitable for challenging datasets where the neuronal structure is contaminated by noise or has interrupted parts. In this paper, we propose a segmentation method based on deep learning to determine the location information of neurons and reduce the influence of image noise in the data. Specifically, we adapt our neuron dataset based on UNet by using convolution with BN fusion and multi-input feature fusion. The method is named REDAFNet. The model simplifies the model structure and enhances the generalization ability by fusing the convolution layer and BN layer. The noise interference in the data was reduced by multi-input feature fusion, and the ability to understand and express the data was enhanced. The method takes a neuron image as input and its pixel segmentation map as output. Experimental results show that the segmentation accuracy of the proposed method is 91.96%, 93.86% and 80.25% on the ISBI2012 dataset, U-RISC retinal neuron dataset and N2DH-GOWT1 stem cell dataset, respectively. Compared with the existing segmentation methods, the proposed method can extract more complete feature information and achieve more accurate segmentation. [ABSTRACT FROM AUTHOR]

Accurate segmentation of knee cartilage in MR images is crucial for early diagnosis and treatment of knee conditions. Manual segmentation is time-consuming, leading researchers to explore automatic deep learning methods. However, the choice between 2D and 3D networks for organ segmentation remains debated. In this paper, we propose a hybrid 2D and 3D deep neural network approach, named UVNet, which combines the strengths of both techniques to enhance segmentation performance. Within this network structure, the 3D segmentation network serves as the backbone for feature extraction, while the 2D segmentation network functions as an information supplement network. Local and global MIP images are generated by employing various maximum intensity projection modes of knee MRI volumes as input for the information supplement network. By constructing a local and global MIP feature fusion module, the supplementary information obtained from the 2D segmentation network is fully integrated into the backbone network. We assess the quality of the proposed method using the Osteoarthritis Initiative (OAI) dataset and the 2010 Grand Challenge Knee Image Segmentation (SKI-10) dataset, comparing it to the Baseline Network and other advanced 2D and 3D segmentation methods. The experiments demonstrate that UVNet achieves competitive performance in the aforementioned two cartilage segmentation tasks. [ABSTRACT FROM AUTHOR]

Accurate and reliable estimation of the pelvic tilt is one of the essential pre-planning factors for total hip arthroplasty to prevent common post-operative complications such as implant impingement and dislocation. Inspired by the latest advances in deep learning-based systems, our focus in this paper has been to present an innovative and accurate method for estimating the functional pelvic tilt (PT) from a standing anterior–posterior (AP) radiography image. We introduce an encoder–decoder-style network based on a concurrent learning approach called VGG-UNET (VGG embedded in U-NET), where a deep fully convolutional network known as VGG is embedded at the encoder part of an image segmentation network, i.e., U-NET. In the bottleneck of the VGG-UNET, in addition to the decoder path, we use another path utilizing light-weight convolutional and fully connected layers to combine all extracted feature maps from the final convolution layer of VGG and thus regress PT. In the test phase, we exclude the decoder path and consider only a single target task i.e., PT estimation. The absolute errors obtained using VGG-UNET, VGG, and Mask R-CNN are 3.04 ± 2.49, 3.92 ± 2.92, and 4.97 ± 3.87, respectively. It is observed that the VGG-UNET leads to a more accurate prediction with a lower standard deviation (STD). Our experimental results demonstrate that the proposed multi-task network leads to a significantly improved performance compared to the best-reported results based on cascaded networks. [ABSTRACT FROM AUTHOR]

Automatic weed identification and detection are crucial for precision weeding operations. In recent years, deep learning (DL) has gained widespread attention for its potential in crop weed identification. This paper provides a review of the current research status and development trends of weed identification in crop fields based on DL. Through an analysis of relevant literature from both within and outside of China, the author summarizes the development history, research progress, and identification and detection methods of DL-based weed identification technology. Emphasis is placed on data sources and DL models applied to different technical tasks. Additionally, the paper discusses the challenges of time-consuming and laborious dataset preparation, poor generality, unbalanced data categories, and low accuracy of field identification in DL for weed identification. Corresponding solutions are proposed to provide a reference for future research directions in weed identification. [ABSTRACT FROM AUTHOR]

The segmentation of dermoscopic‐based skin lesion images is considered to be challenging owing to various factors. Some of the most tangible reasons include poor contrast near the affected skin lesion, the fuzzy and unpredictable lesion limits, the presence of variations in noise, and capturing images under different conditions. This paper aims to develop an efficient segmentation model for dermoscopic images of different skin lesions based on deep learning. This paper proposes the 11‐layer convolutional deep neural network with two segmentation models trained from start to finish and do not depend on any previous information about the data. The viability, efficiency, and speculation ability of the models are evaluated on the ISIC2018 database. The proposed model achieves 0.903 accuracy and 0.820 Jaccard index in the segmentation of skin lesions. The model shows better performance compared to other image segmentation techniques from the leaderboards of ISIC2018 using deep learning. [ABSTRACT FROM AUTHOR]

Background: Functional assessment of right ventricle (RV) using gated myocardial perfusion single‐photon emission computed tomography (MPS) heavily relies on the precise extraction of right ventricular contours. Purpose: In this paper, we present a new deep‐learning‐based model integrating both the spatial and temporal features in gated MPS images to perform the segmentation of the RV epicardium and endocardium. Methods: By integrating the spatial features from each cardiac frame of the gated MPS and the temporal features from the sequential cardiac frames of the gated MPS, we developed a Spatial‐Temporal V‐Net (ST‐VNet) for automatic extraction of RV endocardial and epicardial contours. In the ST‐VNet, a V‐Net is employed to hierarchically extract spatial features, and convolutional long‐term short‐term memory (ConvLSTM) units are added to the skip‐connection pathway to extract the temporal features. The input of the ST‐VNet is ECG‐gated sequential frames of the MPS images and the output is the probability map of the epicardial or endocardial masks. A Dice similarity coefficient (DSC) loss which penalizes the discrepancy between the model prediction and the manual annotation was adopted to optimize the segmentation model. Results: Our segmentation model was trained and validated on a retrospective dataset with 45 subjects, and the cardiac cycle of each subject was divided into eight gates. The proposed ST‐VNet achieved a DSC of 0.8914 and 0.8157 for the RV epicardium and endocardium segmentation, respectively. The mean absolute error, the mean squared error, and the Pearson correlation coefficient of the RV ejection fraction (RVEF) between the manual annotation and the model prediction were 0.0609, 0.0830, and 0.6985. Conclusion: Our proposed ST‐VNet is an effective model for RV segmentation. It has great promise for clinical use in RV functional assessment. [ABSTRACT FROM AUTHOR]

• We introduce a hexagonal convolution neural network for images with hexagonal pixels. • We applied hexCNN to segment spatially resolved transcriptomics data. • HexCNN achieved greater segmentation accuracies than state-of-the-art GNN models. • We noticed considerable dropout events and batch effects hindering further progress. Spatial transcriptomics is a rapidly evolving field that enables researchers to capture comprehensive molecular profiles while preserving information about the physical locations. One major challenge in this research area involves the identification of spatial domains, which are distinct regions characterized by unique gene expression patterns. However, current unsupervised methods have struggled to perform well in this regard due to the presence of high levels of noise and dropout events in spatial transcriptomic profiles. In this paper, we propose a novel hexagonal Convolutional Neural Network (hexCNN) for hexagonal image segmentation on spatially resolved transcriptomics. To address the problem of noise and dropout occurrences within spatial transcriptomics data, we first extend an unsupervised algorithm to a supervised learning method that can identify useful features and reduce noise hindrance. Then, inspired by the classical convolution in convolutional neural networks (CNNs), we designed a regular hexagonal convolution to compensate for the missing gene expression patterns from adjacent spots. We evaluated the performance of hexCNN by applying it to the DLPFC dataset. The results show that hexCNN achieves a classification accuracy of 86.8% and an average Rand index (ARI) of 77.1% (1.4% and 2.5% higher than those of GNNs). The results also demonstrate that hexCNN is capable of removing the noise caused by batch effect while preserving the biological signal differences. [ABSTRACT FROM AUTHOR]

Aiming at the problem that the existing single image super segmentation reconstruction methods are difficult to capture complete and effective information in the image, this paper proposed a joint image-frequency supervised image super-resolution reconstruction algorithm, which aimed to utilize the complementary information between multiple domains, thereby obtaining a more complete image feature representation. Firstly, by analyzing the characteristics of image pixels after converting to frequency domain space, this paper proposed a new frequency domain distance supervision loss according to its complex representation, which effectively applied spectral information to the optimization process of convolutional neural networks. Based on the characterization characteristics of different frequency bands in the domain, this paper constructed the spectral weighting loss on the basis of the distance loss in the frequency domain, and applied it to the two frequency bands of low frequency and high frequency respectively. Finally, combined with the supervision of the image domain, it formed a joint optimization of multiple domains and got good performance. The results on Setl4, BlOO and Kodak show that the PSNR and SSIM of the proposed algorithm reach 33.47 dB and 0.985 9, respectively, and the proposed algorithm achieves the best performance compared with several image super-resolution methods. [ABSTRACT FROM AUTHOR]

The hyperspectral image (HSI) super-resolution reconstruction has attracted much attention and been used widely in various study fields due to its low requirements on hardware in practice. However, most of the hyperspectral image super-resolution reconstruction studies apply one strategy for images with varying complexity of spatial information. This is not conducive to the improvement of Image processing efficiency and the extraction of complex details. Given the above, a new method named MSDESR (multilevel streams and detail enhancement) is proposed to reconstruct HSI by using partition reconstruction and detail enhancement. The MSDESR consists of a sub-map shunt block, a high-low-frequency information extraction with detail enhancement block, and a partition image reconstruction block. Firstly, the sub-map shunt block is designed to pre-classify hyperspectral images. The images are divided into complex and simple parts according to the spatial information distribution of the reconstructed sub-map. Secondly, the multiscale Retinex with detail enhancement algorithm is constructed to purify high-frequency noise-contaminated and enhance the image details by separating the samples into high- and low-frequency information. Finally, branching networks of different complexities are designed to reconstruct the images with high credibility and clear content. In this paper, datasets of QUST-1, Pavia University, Chikusei, the Washington DC Mal and XiongAn are applied in the experiments. The results show that MSDESR outperforms state-of-the-art CNN-based methods in terms of quantitative metrics, visual quality, and computational effort, with a 4.18% and 9.35% improvement in SRE and MPSNR metrics, and a 37% saving in FLOPs. Overall, the MSDESR performs well in hyperspectral image super-resolution reconstruction, which is time saving and preserves the details of spatial information. [ABSTRACT FROM AUTHOR]

Accurate segmentation of lesions can provide strong evidence for early skin cancer diagnosis by doctors, enabling timely treatment of patients and effectively reducing cancer mortality rates. In recent years, some deep learning models have utilized complex modules to improve their performance for skin disease image segmentation. However, limited computational resources have hindered their practical application in clinical environments. To address this challenge, this paper proposes a lightweight model, named GSCEU-Net, which is able to achieve superior skin lesion segmentation performance at a lower cost. GSCEU-Net is based on the U-Net architecture with additional enhancements. Firstly, the partial convolution (PConv) module, proposed by the FasterNet model, is modified to an SConv module, which enables channel segmentation paths of different scales. Secondly, a newly designed Ghost SConv (GSC) module is proposed for incorporation into the model's backbone, where the Separate Convolution (SConv) module is aided by a Multi-Layer Perceptron (MLP) and the output path residuals from the Ghost module. Finally, the Efficient Channel Attention (ECA) mechanism is incorporated at different levels into the decoding part of the model. The segmentation performance of the proposed model is evaluated on two public datasets (ISIC2018 and PH2) and a private dataset. Compared to U-Net, the proposed model achieves an IoU improvement of 0.0261 points and a DSC improvement of 0.0164 points, while reducing the parameter count by 190 times and the computational complexity by 170 times. Compared to other existing segmentation models, the proposed GSCEU-Net model also demonstrates superiority, along with an advanced balance between the number of parameters, complexity, and segmentation performance. [ABSTRACT FROM AUTHOR]

Automatic segmentation of optic disc (OD) and optic cup (OC) is an essential task for analysing colour fundus images. In clinical practice, accurate OD and OC segmentation assist ophthalmologists in diagnosing glaucoma. In this paper, we propose a unified convolutional neural network, named ResFPN-Net, which learns the boundary feature and the inner relation between OD and OC for automatic segmentation. The proposed ResFPN-Net is mainly composed of multi-scale feature extractor, multi-scale segmentation transition and attention pyramid architecture. The multi-scale feature extractor achieved the feature encoding of fundus images and captured the boundary representations. The multi-scale segmentation transition is employed to retain the features of different scales. Moreover, an attention pyramid architecture is proposed to learn rich representations and the mutual connection in the OD and OC. To verify the effectiveness of the proposed method, we conducted extensive experiments on two public datasets. On the Drishti-GS database, we achieved a Dice coefficient of 97.59%, 89.87%, the accuracy of 99.21%, 98.77%, and the Averaged Hausdorff distance of 0.099, 0.882 on the OD and OC segmentation, respectively. We achieved a Dice coefficient of 96.41%, 83.91%, the accuracy of 99.30%, 99.24%, and the Averaged Hausdorff distance of 0.166, 1.210 on the RIM-ONE database for OD and OC segmentation, respectively. Comprehensive results show that the proposed method outperforms other competitive OD and OC segmentation methods and appears more adaptable in cross-dataset scenarios. The introduced multi-scale loss function achieved significantly lower training loss and higher accuracy compared with other loss functions. Furthermore, the proposed method is further validated in OC to OD ratio calculation task and achieved the best MAE of 0.0499 and 0.0630 on the Drishti-GS and RIM-ONE datasets, respectively. Finally, we evaluated the effectiveness of the glaucoma screening on Drishti-GS and RIM-ONE datasets, achieving the AUC of 0.8947 and 0.7964. These results proved that the proposed ResFPN-Net is effective in analysing fundus images for glaucoma screening and can be applied in other relative biomedical image segmentation applications. [ABSTRACT FROM AUTHOR]

As we know, image colorization is widely used in computer graphics and has become a research hotspot in the field of image processing. Current image colorization technology has the phenomenon of single coloring effect and unreal color, which is too complicated to be implemented and struggled to gain popularity. In this paper, a new method based on a convolution neural network is proposed to study the reasonable coloring of human images and ensures the realism of the coloring effect and the diversity of coloring at the same time. First, this paper selects about 5000 pictures of people and plants from the Imagenet dataset and makes a small dataset containing only people and backgrounds. Secondly, in order to obtain the image segmentation results, this paper improves the U-net network and carries out three times of down sampling and three times of up-sampling. Finally, we add the expanded convolution, and use the sigmoid activation function to replace the ReLU (The Rectified Linear Unit) activation function and put the BN (Batch Normalization) before the activation function. Experimental results show that our proposed image colorization algorithm based on the deep learning scheme can reduce the training time of the network and achieve higher quality segmentation results. [ABSTRACT FROM AUTHOR]

In recent years, the convolutional segmentation network has achieved remarkable performance in the computer vision area. However, training a practicable segmentation network is time- and resource-consuming. In this paper, focusing on the semantic image segmentation task, we attempt to disassemble a convolutional segmentation network into category-aware convolution kernels and achieve customizable tasks without additional training by utilizing those kernels. The core of disassembling convolutional segmentation networks is how to identify the relevant convolution kernels for a specific category. According to the encoder-decoder network architecture, the disassembling framework, named Disassembler, is devised to be composed of the forward channel-wise activation attribution and backward gradient attribution. In the forward channel-wise activation attribution process, for each image, the activation values of each feature map in the high-confidence mask area are summed into category-aware probability vectors. In the backward gradient attribution process, the positive gradients w.r.t. each feature map in the high-confidence mask area are summed into a relative coefficient vector for each category. With the cooperation of two vectors, the Disassembler can effectively disassemble category-aware convolution kernels. Extensive experiments demonstrate that the proposed Disassembler can accomplish the category-customizable task without additional training. The disassembled category-aware sub-network achieves comparable performance without any finetuning and will outperform existing state-of-the-art methods with one epoch of finetuning. [ABSTRACT FROM AUTHOR]

Accurate and effective semantic segmentation methods for remote sensing are important for applications such as precision agriculture, urban planning, and disaster monitoring. Convolutional neural networks (CNN) have achieved remarkable performance in the field of remote sensing semantic segmentation. However, the design of CNNs is both time-consuming and necessitates a substantial amount of domain expertise and experience. To address the aforementioned issues, we propose a neural architecture search method called EAS-CNN. The method constructs a search space based on a U-shaped structure and utilizes a fixed-length encoding solution based on gene expression suppression to preserve potential useful information during the evolution process. Furthermore, an improved genetic strategy is proposed to enhance search efficiency and save computational resources. In this paper, we evaluate the proposed EAS-CNN method against state-of-the-art semantic segmentation methods and verify its effectiveness. Experimental results show that EAS-CNN achieves high OA values of 91.2% and 91.6% on the Vaihingen and Postman datasets, respectively. Furthermore, we conduct a thorough analysis of the experimental results and summarize effective design patterns for model architecture to enhance remote sensing semantic segmentation tasks. [ABSTRACT FROM AUTHOR]

Automated medical image segmentation plays an important role in many clinical applications, which however is a very challenging task, due to complex background texture, lack of clear boundary and significant shape and texture variation between images. Many researchers proposed an encoder–decoder architecture with skip connections to combine low-level feature maps from the encoder path with high-level feature maps from the decoder path for automatically segmenting medical images. The skip connections have been shown to be effective in recovering fine-grained details of the target objects and may facilitate the gradient back-propagation. However, not all the feature maps transmitted by those connections contribute positively to the network performance. In this paper, to adaptively select useful information to pass through those skip connections, we propose a novel 3D network with self-supervised function, named selective information passing network. We evaluate our proposed model on the MICCAI Prostate MR Image Segmentation 2012 Grant Challenge dataset, TCIA Pancreas CT-82 and MICCAI 2017 Liver Tumor Segmentation Challenge dataset. The experimental results across these datasets show that our model achieved improved segmentation results and outperformed other state-of-the-art methods. The source code of this work is available at https://github.com/ahukui/SIPNet. [ABSTRACT FROM AUTHOR]

Accurate medical imaging segmentation of the retinal fundus vasculature is essential to assist physicians in diagnosis and treatment. In recent years, convolutional neural networks (CNN) are widely used to classify retinal blood vessel pixels for retinal blood vessel segmentation tasks. However, the convolutional block receptive field is limited, simple multiple superpositions tend to cause information loss, and there are limitations in feature extraction as well as vessel segmentation. To address these problems, this paper proposes a new retinal vessel segmentation network based on U-Net, which is called multi-scale cross-position attention network (MCPANet). MCPANet uses multiple scales of input to compensate for image detail information and applies to skip connections between encoding blocks and decoding blocks to ensure information transfer while effectively reducing noise. We propose a cross-position attention module to link the positional relationships between pixels and obtain global contextual information, which enables the model to segment not only the fine capillaries but also clear vessel edges. At the same time, multiple scale pooling operations are used to expand the receptive field and enhance feature extraction. It further reduces pixel classification errors and eases the segmentation difficulty caused by the asymmetry of fundus blood vessel distribution. We trained and validated our proposed model on three publicly available datasets, DRIVE, CHASE, and STARE, which obtained segmentation accuracy of 97.05%, 97.58%, and 97.68%, and Dice of 83.15%, 81.48%, and 85.05%, respectively. The results demonstrate that the proposed method in this paper achieves better results in terms of performance and segmentation results when compared with existing methods. [ABSTRACT FROM AUTHOR]

Retinal vessel segmentation is extremely important for risk prediction and treatment of many major diseases. Therefore, accurate segmentation of blood vessel features from retinal images can help assist physicians in diagnosis and treatment. Convolutional neural networks are good at extracting local feature information, but the convolutional block receptive field is limited. Transformer, on the other hand, performs well in modeling long-distance dependencies. Therefore, in this paper, a new network model MTPA_Unet is designed from the perspective of extracting connections between local detailed features and making complements using long-distance dependency information, which is applied to the retinal vessel segmentation task. MTPA_Unet uses multi-resolution image input to enable the network to extract information at different levels. The proposed TPA module not only captures long-distance dependencies, but also focuses on the location information of the vessel pixels to facilitate capillary segmentation. The Transformer is combined with the convolutional neural network in a serial approach, and the original MSA module is replaced by the TPA module to achieve finer segmentation. Finally, the network model is evaluated and analyzed on three recognized retinal image datasets DRIVE, CHASE DB1, and STARE. The evaluation metrics were 0.9718, 0.9762, and 0.9773 for accuracy; 0.8410, 0.8437, and 0.8938 for sensitivity; and 0.8318, 0.8164, and 0.8557 for Dice coefficient. Compared with existing retinal image segmentation methods, the proposed method in this paper achieved better vessel segmentation in all of the publicly available fundus datasets tested performance and results. [ABSTRACT FROM AUTHOR]

High-resolution remote sensing images have become mainstream remote sensing data, but there is an obvious "salt and pepper phenomenon" in the existing semantic segmentation methods of high-resolution remote sensing images. The purpose of this paper is to propose an improved deep convolutional neural network based on HRNet and PSPNet to segment and realize deep scene analysis and improve the pixel-level semantic segmentation representation of high-resolution remote sensing images. Based on hierarchical multiscale segmentation technology research, the main method is multiband segmentation; the vegetation, buildings, roads, waters and bare land rule sets in the experimental area are established, the classification is extracted, and the category is labeled at each pixel in the image. Using the image classification network structure, different levels of feature vectors can be used to meet the judgment requirements. The HRNet and PSPNet algorithms are used to analyze the scene and obtain the category labels of all pixels in an image. Experiments have shown that artificial intelligence uses the pyramid pooling module in the classification and recognition of CCF satellite images. In the context of integrating different regions, PSPNet affects the region segmentation accuracy. FCN, DeepLab and PSPNet are now the best methods and achieve 98% accuracy. However, the PSPNet object recognition algorithm has better advantages in specific areas. Experiments show that this method has high segmentation accuracy and good generalization ability and can be used in practical engineering. [ABSTRACT FROM AUTHOR]

Active contour models (ACMs) are popular and widely used for many image segmentation applications and obtain promising results. However, these methods are unable to achieve the highest performance in the presence of intensity inhomogeneity. To address this issue, this paper presents an ACM based on the combination of a convolutional neural network (CNN) and a texture descriptor approach. This study uses a CNN model to generate parameter maps more effectively for ACM. Compared to conventional global techniques, these parameter maps increase the speed of movement of the contour into the target. In this approach, the Local Word Directional Pattern (LWDP) is applied as the texture descriptor. LWDP is a texture descriptor that uses the angle between two gradients for exploring the texture structure inside the image. In the proposed method both the original image and the new image obtained by the LWDP texture descriptor (encoded image) are provided as inputs to the CNN. The experimental outcomes show that the proposed strategy consistently outperforms the state-of-the-art in accuracy and robustness for segmenting images with fuzzy boundaries and intensity inhomogeneity. [ABSTRACT FROM AUTHOR]

Aim: The main objective of this work is to propose an efficient segmentation model for accurate and robust lung segmentation from computed tomography (CT) images, even when the lung contains abnormalities such as juxtapleural nodules, cavities, and consolidation. Methodology: A novel deep learning‑based segmentation model, pyramid‑dilated dense U‑Net (PDD‑U‑Net), is proposed to directly segment lung regions from the whole CT image. The model is integrated with pyramid‑dilated convolution blocks to capture and preserve multi‑resolution spatial features effectively. In addition, shallow and deeper stream features are embedded in the nested U‑Net structure at the decoder side to enhance the segmented output. The effect of three loss functions is investigated in this paper, as the medical image analysis method requires precise boundaries. The proposed PDD‑U‑Net model with shape‑aware loss function is tested on the lung CT segmentation challenge (LCTSC) dataset with standard lung CT images and the lung image database consortium‑image database resource initiative (LIDC‑IDRI) dataset containing both typical and pathological lung CT images. Results: The performance of the proposed method is evaluated using Intersection over Union, dice coefficient, precision, recall, and average Hausdorff distance metrics. Segmentation results showed that the proposed PDD‑U‑Net model outperformed other segmentation methods and achieved a 0.983 dice coefficient for the LIDC‑IDRI dataset and a 0.994 dice coefficient for the LCTSC dataset. Conclusions: The proposed PDD‑U‑Net model with shape‑aware loss function is an effective and accurate method for lung segmentation from CT images, even in the presence of abnormalities such as cavities, consolidation, and nodules. The model’s integration of pyramid‑dilated convolution blocks and nested U‑Net structure at the decoder side, along with shape‑aware loss function, contributed to its high segmentation accuracy. This method could have significant implications for the computer‑aided diagnosis system, allowing for quick and accurate analysis of lung regions. [ABSTRACT FROM AUTHOR]

Chronic wounds, are a worldwide health problem affecting populations and economies as a whole. With the increase in age-related diseases, obesity, and diabetes, the costs of chronic wound healing will further increase. Wound assessment should be fast and accurate in order to reduce possible complications and thus shorten the wound healing process. This paper describes an automatic wound segmentation based on a wound recording system built upon a 7-DoF robot arm with an attached RGB-D camera and high-precision 3D scanner. The developed system represents a novel combination of 2D and 3D segmentation, where the 2D segmentation is based on the MobileNetV2 classifier and the 3D component is based on the active contour model, which works on the 3D mesh to further refine the wound contour. The end output is the 3D model of only the wound surface without the surrounding healthy skin and geometric parameters in the form of perimeter, area, and volume. [ABSTRACT FROM AUTHOR]

The task of cell segmentation in microscope images is difficult and popular. In recent years, deep learning-based techniques have made incredible progress in medical and microscopy image segmentation applications. In this paper, we propose a novel deep learning approach called Residual-Atrous MultiResUnet with Channel Attention Mechanism (RAMRU-CAM) for cell segmentation, which combines MultiResUnet architecture with Channel Attention Mechanism (CAM) and Residual-Atrous connections. The Residual-Atrous path mitigates the semantic gap between the encoder and decoder stages and manages the spatial dimension of feature maps. Furthermore, the Channel Attention Mechanism (CAM) blocks are used in the decoder stages to better maintain the spatial details before concatenating the feature maps from the encoder phases to the decoder phases. We evaluated our proposed model on the PhC-C2DH-U373 and Fluo-N2DH-GOWT1 datasets. The experimental results show that our proposed model outperforms recent variants of the U-Net model and the state-of-the-art approaches. We have demonstrated how our model can segment cells precisely while using fewer parameters and low computational complexity. [ABSTRACT FROM AUTHOR]

Background: Automatic segmentation of lesion, organ, and tissue from the medical image is an important part of medical image analysis, which are useful for improving the accuracy of disease diagnosis and clinical analysis. For skin melanomas lesions, the contrast ratio between lesions and surrounding skin is low and there are many irregular shapes, uneven distribution, and local and boundary features. Moreover, some hair covering the lesions destroys the local context. Polyp characteristics such as shape, size, and appearance vary at different development stages. Early polyps with small sizes have no distinctive features and could be easily mistaken for other intestinal structures, such as wrinkles and folds. Imaging positions and illumination conditions would alter polyps’ appearance and lead to no visible transitions between polyps and surrounding tissue. It remains a challenging task to accurately segment the skin lesions and polyps due to the high variability in the location, shape, size, color, and texture of the target object.Developing a robust and accurate segmentation method for medical images is necessary. Purpose: To achieve better segmentation performance while dealing with the difficulties above, a U-shape network based on the encoder and decoder structure is proposed to enhance the segmentation performance in target regions. Methods: In this paper,a novel deep network of the encoder-decoder model that combines HarDNet, dual attention (DA),and reverse attention (RA) is proposed. First, HarDNet68 is employed to extract the backbone features while improving the inference speed and computational efficiency. Second, the DA block is adopted to capture the global feature dependency in spatial and channel dimensions, and enrich the contextual information on local features. At last, three RA blocks are exploited to fuse and refine the boundary features to obtain the final segmentation results. Results: Extensive experiments are conducted on a skin lesion dataset which consists of ISIC2016, ISIC2017, and ISIC 2018, and a polyp dataset which consists of several public datasets, that is, Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, Endosece. The proposed method outperforms some state-of-art segmentation models on the ISIC2018,ISIC2017,and ISIC2016 datasets, with Jaccard’s indexes of 0.846, 0.881, and 0.894,mean Dice coefficients of 0.907, 0.929, and 03939, precisions of 0.908, 0.977, and 0.968, and accuracies of 0.953, 0.975, and 0.972. Additionally, the proposed method also performs better than some state-of-art segmentation models on the Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, and Endosece datasets, with mean Dice coefficients of 0.907, 0.935, 0.716, 0.667, and 0.887, mean intersection over union coefficients of 0.850, 0.885, 0.644, 0.595, and 0.821, structural similarity measures of 0.918, 0.953, 0.823, 0.807, and 0.933, enhanced alignment measures of 0.952, 0.983, 0.850, 0.817,and 0.957, mean absolute errors of 0.026, 0.007, 0.037, 0.030,and 0.009. Conclusions: The proposed deep network could improve lesion segmentation performance in polyp and skin lesion images. The quantitative and qualitative results show that the proposed method can effectively handle the challenging task of segmentation while revealing the great potential for clinical application. [ABSTRACT FROM AUTHOR]

Automatic crack detection is always a challenging task due to the inherent complex backgrounds, uneven illumination, irregular patterns, and various types of noise interference. In this paper, we proposed a U-shaped encoder–decoder semantic segmentation network combining Unet and Resnet for pixel-level pavement crack image segmentation, which is called RUC-Net. We introduced the spatial-channel squeeze and excitation (scSE) attention module to improve the detection effect and used the focal loss function to deal with the class imbalance problem in the pavement crack segmentation task. We evaluated our methods using three public datasets, CFD, Crack500, and DeepCrack, and all achieved superior results to those of FCN, Unet, and SegNet. In addition, taking the CFD dataset as an example, we performed ablation studies and compared the differences of various scSE modules and their combinations in improving the performance of crack detection. [ABSTRACT FROM AUTHOR]

A polyp is one of the major causes of gastroenterology, which leads to colorectal cancer. The detection of polyps by colonoscopy imaging is a significant challenge because of the diversity in polyp structure and lack of examination accuracy. To solve this problem, the automatic segmentation of polyps can be used to enhance examination accuracy and reduce gastrointestinal (GI) disease. In this paper, the framework of polyp image segmentation is developed by a deep learning approach, especially a convolutional neural network. This proposed framework used the Kvasir‐SEG database, which contains 1000 GI polyp images and corresponding segmentation masks according to annotation by medical experts. This database is divided into 900 for training images and 100 for testing images. This framework is based on image preprocessing and two types of SegNet architecture to obtain the segmented polyp image. This paper has demonstrated state‐of‐the‐art performance on both VGG‐16, and VGG‐19 networks for training and testing data to address colorectal cancer screening rates. The results confirmed that the VGG‐19 model has outperformed the VGG‐16 model via all evaluation parameters except sensitivity for the polyp segmentation on the Kvasir‐SEG dataset. Additionally, it will support gastroenterologists during medical strategy to correctly choose the treatment with less time. [ABSTRACT FROM AUTHOR]

With the leaps and bounds of computer performance and the advent of the era of big data, deep learning has drawn more and more attention from all walks of life. It can combine low-level features to form more abstract high-level features and describe the data more essentially. Therefore, it is widely used in various fields such as computer vision. Image segmentation is one of the most basic research topics in the field of computer vision. The main purpose is to extract regions of interest from images for later image processing research. In 3D reconstruction based on sequence images, the segmentation accuracy and speed of sequence images determine the quality and efficiency of target reconstruction. Therefore, when facing large-scale sequence images, the biggest problem is how to improve the segmentation speed while ensuring accuracy. Based on the above background, the research content of this article is an indoor scene segmentation algorithm based on full convolutional neural network. According to the characteristics of indoor application scenes, this paper proposes a fast convolutional neural network image segmentation method to segment the indoor scene image and construct the fast fully convolutional networks (FFCN) for indoor scene image segmentation uses inter-layer fusion to reduce the amount of network calculation parameters and avoid the loss of picture feature information by continuous convolution. In order to verify the effectiveness of the network, in this paper, a basic living object data set (XAUT data set) in an indoor environment is created. The XAUT data set is used to train the FFCN network under the Caffe framework to obtain an indoor scene segmentation model. In order to compare the effectiveness of the model, the structure of the worn FCN8s, FCN16s, and FCN32s models was fine-tuned, and the corresponding algorithm model for indoor scene segmentation was obtained by training with the XAUT data set. The experimental results show that the pixel recognition accuracy of all types of networks has reached 86%, and the mean IU ratio has reached more than 63%. The mean IU of the FCN8s network is the highest at 70.38%, but its segmentation speed is only 1/5 of FFCN. On the premise that other types of indicators are not much different, the average segmentation speed on FFCN fast segmentation convolutional neural network reaches 40 fps. It can be seen that the scale fusion technology can well avoid the loss of image feature information in the network convolution and reddening process. Compared with other FCN networks, it has a faster speed and is conducive to real-time image preprocessing. [ABSTRACT FROM AUTHOR]

Due to the complexity of medical imaging techniques and the high heterogeneity of glioma surfaces, image segmentation of human gliomas is one of the most challenging tasks in medical image analysis. Current methods based on convolutional neural networks concentrate on feature extraction while ignoring the correlation between local and global. In this paper, we propose a residual mix transformer fusion net, namely RMTF-Net, for brain tumor segmentation. In the feature encoder, a residual mix transformer encoder including a mix transformer and a residual convolutional neural network (RCNN) is proposed. The mix transformer gives an overlapping patch embedding mechanism to cope with the loss of patch boundary information. Moreover, a parallel fusion strategy based on RCNN is utilized to obtain local–global balanced information. In the feature decoder, a global feature integration (GFI) module is applied, which can enrich the context with the global attention feature. Extensive experiments on brain tumor segmentation from LGG, BraTS2019 and BraTS2020 demonstrated that our proposed RMTF-Net is superior to existing state-of-art methods in subjective visual performance and objective evaluation. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

Regular patrol inspection of concrete dams can detect cracks at an early stage. However, conventional crack segmentation models based on deep learning (DL) are difficult to be deployed in resource-constrained mobile devices due to the large number of parameters. This paper describes a lightweight semantic segmentation model, termed as CrackTrNet, for images of concrete dam cracks. CrackTrNet is a hybrid U-shaped model based on convolutional neural network (CNN) and Vision Transformer. The CNN is adopted to extract low-level visual features and the Transformer focuses on learning the global contextual information. The results demonstrate that its segmentation accuracy can reach 97.60%, while the model size is only 34.86 MB, which is 66.12%–87.85% lower than that of current mainstream DL-based models. To make the model more practical, a crack inspection mobile application (APP) is developed using Android Studio. The integration of lightweight CrackTrNet and APP can effectively assist the intelligent inspection of dam cracks to ensure structural safety. • A lightweight hybrid network, CrackTrNet, is presented for semantic segmentation of dam cracks. • The integration of CNN and Transformer maximizes the preservation of global and local features. • Transfer learning and depthwise separable convolution are introduced to optimize the model size. • The mobile application is developed to assist patrol inspection of cracks in concrete dams. [ABSTRACT FROM AUTHOR]

Pavement crack detection continues to be a stubborn problem given the interference of various factors in the actual pavement and the complex topological structure of asphalt pavement. Among all the obstacles, the bottleneck of pavement crack detection lies in the difficulty of segmenting the cracks in the pavement images whose edges are blurred. This paper proposes a multi-scale context feature and cross-attention based on convolutional neural network for accurate and robust pavement crack segmentation. The multi-scale context feature module is built in different deep networks to extract rich crack feature information. Subsequently, in order to effectively promote the seamless integration of features at different levels, we deploy cross-attention modules to each branch. After that, we add deep supervision to each branch to accelerate training. Finally, we integrate the outputs of each branch to obtain the final output diagram. The comparative experiments on various pavement datasets show that the method has better robustness. At the same time, this paper designs a complete system of pavement crack detection (PCD) and develops corresponding engineering application software. The PCD system can record the real-time pavement image data to the edge server, and the client can also monitor the real-time pavement images from the edge server through the HTTP protocol. • A multi-scale context feature module is built in the deep layer for robustly describing the size of the cracks. • The cross-attention module enables each pixel to select individual contextual information from multi-layer features. • In-depth supervision is added to each branch to accelerate the optimization of the network model. • An intelligent detection application software is designed and integrated with multiple modules. [ABSTRACT FROM AUTHOR]

OCT fluid segmentation is a crucial task for diagnosis and therapy in ophthalmology. The current convolutional neural networks (CNNs) supervised by pixel-wise annotated masks achieve great success in OCT fluid segmentation. However, requiring pixel-wise masks from OCT images is time-consuming, expensive and expertise needed. This paper proposes an Intra- and inter-Slice Contrastive Learning Network (ISCLNet) for OCT fluid segmentation with only point supervision. Our ISCLNet learns visual representation by designing contrastive tasks that exploit the inherent similarity or dissimilarity from unlabeled OCT data. Specifically, we propose an intra-slice contrastive learning strategy to leverage the fluid-background similarity and the retinal layer-background dissimilarity. Moreover, we construct an inter-slice contrastive learning architecture to learn the similarity of adjacent OCT slices from one OCT volume. Finally, an end-to-end model combining intra- and inter-slice contrastive learning processes learns to segment fluid under the point supervision. The experimental results on two public OCT fluid segmentation datasets (i.e., AI Challenger and RETOUCH) demonstrate that the ISCLNet bridges the gap between fully-supervised and weakly-supervised OCT fluid segmentation and outperforms other well-known point-supervised segmentation methods. [ABSTRACT FROM AUTHOR]

Medical image segmentation plays a vital role in disease diagnosis and analysis. However, data-dependent difficulties such as low image contrast, noisy background, and complicated objects of interest render the segmentation problem challenging. These difficulties diminish dense prediction and make it tough for known approaches to explore data-specific attributes for robust feature extraction. In this paper, we study medical image segmentation by focusing on robust data-specific feature extraction to achieve improved dense prediction. We propose a new deep convolutional neural network (CNN), which exploits specific attributes of input datasets to utilize deep supervision for enhanced feature extraction. In particular, we strategically locate and deploy auxiliary supervision, by matching the object perceptive field (OPF) (which we define and compute) with the layer-wise effective receptive fields (LERF) of the network. This helps the model pay close attention to some distinct input data dependent features, which the network might otherwise ‘ignore’ during training. Further, to achieve better target localization and refined dense prediction, we propose the densely decoded networks (DDN), by selectively introducing additional network connections (the ‘crutch’ connections). Using five public datasets (two retinal vessel, melanoma, optic disc/cup, and spleen segmentation) and two in-house datasets (lymph node and fungus segmentation), we verify the effectiveness of our proposed approach in 2D and 3D segmentation. [ABSTRACT FROM AUTHOR]

Deep learning has been widely applied to the recognition and segmentation of plant leaf disease. However, traditional recognition models of plant leaf disease usually lack transparency, since these end-to-end deep classifiers are susceptible to shadow, occlusion, and light intensity. To address this drawback, in this paper, we proposed a new recognition and segmentation model of plant leaf disease based on deconvolution-guided, called Deconvolution-Guided VGGNet (DGVGGNet). An encoder-decoder architecture with symmetric convolutional-deconvolutional layers was applied to DGVGGNet so that the plant leaf disease recognition and segmentation can be carried simultaneously. Our model consists of three main phases: recognition, inversion, and deconvolution. In the recognition phase, we first fed the VGGNet with plenty of plant leaf disease images, and then Categorical Cross-Entropy Loss was utilized to train the recognition model. VGGNet was made up of the convolution layers of VGG16 and 2 fully connected layers, and the weights of convolution layers were pre-trained on ImageNet. Ten kinds of tomato leaf disease images in the PlantVillage dataset were used in this paper, concretely, 30% of the pictures were used for training, and the rest 70% were used for the test. Besides, on-the-fly data augmentation was also exploited during the training stage, such as flipping the images and corrupting the origin images by brightness, saturation, salt noise, and Gaussian noise. In the inversion phase, two fully connected layers were used to reverse the category prediction vector. Simultaneously, two skip connections were used to reinforce the decoder by adding the vector of the VGGNet’s fully connected layers. Then, the feature vector was reshaped to feature maps to input into the deconvolution module. In the deconvolution phase, the feature maps were fed into the deconvolution module to acquire the segmentation result of the disease area, where each pixel was trained with the Binary Cross-Entropy Loss. The deconvolution module consists of the upsample and convolution operation. Meanwhile, five skip connections were used to fuse the multiple features, which can refine the segmentation results. Besides, only a few samples of the training set were given pixel-level labels of plant disease to supervise the output of the deconvolution module. At the end of the deconvolution module, the reconstruction layer was used to smooth the segmentation edge. To explore the influence of the number of pixel-level labels used in the model, 9 and 45 pixel-level labels were used to supervise the segmentation results, respectively. To simulate the natural conditions, different kinds of interference were added to the test data, such as translation, fruit occlusion, soil occlusion, leaf occlusion, and brightness reduction with different percentages. Experimentally, we evaluate our recognition module by exploring the performance of VGGNet, DGVGGNet-9, and DGVGGNet-45 on different interference datasets, respectively. We also evaluate our deconvolution module by exploring 4 different evaluation metrics, i.e., PA, MPA, MIoU, and FWIoU, compared with 3 popular semantic segmentation models, i.e., FCN-8s, U-Net, and SegNet. The results show that DGVGGNet-45 has the highest recognition accuracy as well as with the highest PA, MIoU, and FWIoU among the four segmentation evaluation metrics, which are 94.66%, 75.36%, and 90.46%, respectively. Compared with VGGNet, the deconvolution module of DGVGGNet-45 can guide the recognition module to pay more attention to the actual area of disease, which is effective in improving the segmentation accuracy. The recognition results demonstrate that DGVGGNet had strong robustness in tough conditions. Furthermore, DGVGGNet only took 12 ms to identify a single picture on the GPU, which can meet the real-time requirements. [ABSTRACT FROM AUTHOR]

Accurate fault identification provides an important basis for well location deployment, oil and gas resource development. However, obtaining a large number of fault samples through manual labeling or interpretation is difficult. Conventional convolutional neural networks have certain challenges in finely characterizing low-order faults and fault continuity. This paper studies a 3D intelligent fault identification method that integrates deep learning with attention mechanism. Based on 3D-UNet network, the method constructs a dual attention fault identification model with an encoder-decoder structure, which focuses on enhancing low-order fault information and extracting key features. An adaptive hybrid loss function is proposed to address the imbalance problem of fault proportions in seismic amplitude data. The function introduces an adjustment coefficient to adaptively enhance the attention to high-order or low-order faults according to the actual situation of the fault, improving the accuracy and continuity of fault identification. Furthermore, a model re-training learning optimization method is presented to capture the missed fault information in the first training and repair the connectivity of low-order faults. Applying this method to the synthetic data set, the accuracy can be improved to 0.9732, and the loss can be converged to 0.0982. The results of shale reservoirs in the North Sea Basin and Kerry-3D indicate that the method can reduce the situation of fault missing and misidentification, effectively improve fault identification accuracy, and enhance low-order fault identification by obtaining more contextual fault features. • A seismic fault image segmentation network is developed with attention mechanism. • Construct an adaptive hybrid loss function to solve the problem of data imbalance. • A method retraining learning that integrates theory and predictive data is proposed. [ABSTRACT FROM AUTHOR]

The detection and classification of concrete damage is essential for maintaining good infrastructure condition. Traditional semantic segmentation methods often can not provide accurate crack boundary information, which limits the further location and measurement analysis. In this study, the case segmentation method is used to solve the shortcomings of the previous detection methods and achieve more accurate crack identification results. This paper presents an improved YOLOv7 network design scheme. The network includes three different custom modules that can optimize the algorithm to solve missing feature problems, small recognition frames, and gradient problems, thereby improving accuracy. In addition, data sets with different sizes, exposures and noise are used to train the network, which expands the prediction range of the network and enhances the stability of the network. The experimental results show that compared with YOLOv7, YOLOv5, SOLOv2, Cascade Mask R-CNN, Condinst, Sparseinst, mAP is significantly improved. Thus, the proposed network algorithm has high practical engineering value. • Crack morphology detection and category classification are achieved using instance segmentation. • Two innovative modules and a unique connection are developed. • A dataset for crack with high exposure, darkness, and noise is constructed. • The model achieves mAP of 72.2% and 87.9% on public and self-built datasets. [ABSTRACT FROM AUTHOR]

Ultrasound assessment of fetal biometry is widely used for gestational age prediction and abnormal fetal growth diagnosis. However, manual biometric measurements require time-consuming procedures and current automated measurement approaches remain challenging in terms of accuracy and computational complexity. This paper proposes deep learning-based efficient automatic fetal biometry measurement method for system-on-chip (SoC) solution. For end-to-end automated measurements, a hardware-friendly bilateral segmentation network (H-BiSeNet) was designed and optimized for multiple fetal objects (i.e., head, abdomen and femur), and fetal biometric parameters were then calculated with a robust multiple keypoint detection algorithm. For hardware implementation with a SoC, a deep learning processing unit (DPU) was employed to accelerate the proposed segmentation network, and biometric measurements with image pre- and postprocessing were conducted on an application processing unit (APU). A total of 6000 fetal ultrasound images in the three regions were collected from 2568 subjects, and the dataset was utilized to train the proposed network. In the performance evaluation, the proposed segmentation network with optimal quantization (H-BiSeNet-Q) outperformed other segmentation networks (i.e., U-Net, U-Net++, Attention U-Net, U-Net3+, DeepLabv3 + and BiSeNetv2) in terms of the average Dice coefficient, and automated measurements with H-BiSeNet-Q were highly correlated with manual measurements (r ≥ 0.983, p < 0.01 for all parameters, i.e., biparietal diameter, head circumference, abdominal circumference and femur length). In the execution time analysis, H-BiSeNet-Q ran at 4.25 frames per second (FPS) with parallelism corresponding to 80% resource utilization in the SoC. These results showed feasibility in low-resource hardware settings such as portable ultrasound systems. [ABSTRACT FROM AUTHOR]

• A new CNN structure is designed to extract two complementary features of image. • A new feature is used to remedy the missing image details in semantic feature. • A full-feature extraction module is designed to extract the new feature. • A fusion module is used to aggregate image features selectively and effectively. Convolutional neural networks (CNN) has been widely used in biomedical image segmentation (BIS) tasks for its remarkable feature representation capability, and most of existing CNN-based segmentation networks leverage a down-sampling operation to achieve larger acceptance domain. However, down-sampling operations could inevitably loss the detailed information of images which is very important for the BIS task. In this paper, we propose a full-resolution biomedical image segmentation network(FRNet) that could maintain the integrated detailed information of image while keeping sufficient semantic information and large receptive field. Specifically, the basic semantic feature and non-destructive feature are employed to represent the semantic and detailed information of images, respectively. A backbone network and a new full-feature extraction branch are conducted to extract those two kinds of complementary features. Furthermore, a novel feature fusion module is designed to integrate those complementary features to achieve non-destructive description of images. Finally, in order to further improve the description ability of the integrated feature, a Densely connected Atrous Spatial Pyramid Pooling(DenseASPP) module is arranged at the end of our proposed FRNet to extract the multiscale information of images. Thorough experimental results on several available databases demonstrate the effectiveness and advancement of FRNet. [ABSTRACT FROM AUTHOR]

Remote sensing techniques have been developed over the past decades to acquire data without being in contact of the target object or data source. Their application on land-cover image segmentation has attracted significant attention during recent years. With the help of satellites, scientists and researchers can collect and store high-resolution image data that can be further be processed, segmented, and classified. However, these research results have not yet been synthesized to provide coherent guidance on the effect of variant land-cover segmentation processes. In this paper, we present a novel model that augments segmentation using smaller networks to segment individual classes. The combined network is trained on the same data but with the masks, combined and trained using categorical cross entropy. Experimental results show that the proposed method produces the highest mean IoU (Intersection of Union) as compared against several existing state-of-the-art models on the DeepGlobe dataset. [ABSTRACT FROM AUTHOR]

This paper discusses how to efficiently recognize flowers based on a convolutional neural network (CNN) using multiple features. Our proposed work consists of three phases including segmentation by Otsu thresholding with particle swarm optimization algorithms, feature extraction of color, shape, texture and recognition with the LeNet-5 neural network. In the feature extraction, an improved H component with the definition of WGB value is applied to extract the color feature, and a new algorithm based on local binary pattern (LBP) is proposed to enhance the accuracy of texture extraction. Besides this, we replace ReLU with Mish as activation function in the network design, and therefore increase the accuracy by 8% accuracy according to our comparison. The Oxford-102 and Oxford-17 datasets are adopted for benchmarking. The experimental results show that the combination of color features and texture features generates the highest recognition accuracy as 92.56% on Oxford-102 and 93% on Oxford-17. [ABSTRACT FROM AUTHOR]

The encoding of a message is the creation of the message. The decoding of a message is how people can comprehend, and decipher the message. It is a procedure of understanding and interpretation of coded data into a comprehensible form. In this paper, a self-created explicitly defined function for encoding numerical digits into graphical representation is proposed. The proposed system integrates deep learning methods to get the probabilities of digit occurrence and Edge detection techniques for decoding the graphically encoded numerical digits to numerical digits as text. The proposed system's major objective is to take in an Image with digits encoded in graphical format and give the decoded stream of digits corresponding to the graph. This system also employs relevant pre-processing techniques to convert RGB to text and image to Canny image. Techniques such as Multi-Label Classification of images and Segmentation are used for getting the probability of occurrence. The dataset is created, on our own, that consists of 1000 images. The dataset has the training data and testing data in the proportion of 9 : 1. The proposed system was trained on 900 images and the testing was performed on 100 images which were ordered in 10 classes. The model has created a precision of 89% for probability prediction. [ABSTRACT FROM AUTHOR]

In this paper, we proposed a semantic fire image segmentation method using a convolutional neural network. The simple but powerful method proposed is middle skip connection achieved through the residual network, which is widely used in image-based deep learning. To enhance the middle skip connection, we constructed a pair of convolution layers, hereafter referred to as input convolution and output convolution, to be inserted in front and behind of the entire architecture. Consequently, the middle skip connection yields a stronger feedback effect compared to when only the short skip connection of the residual block and the long skip connection are used. The validity of the proposed method has been confirmed by using the FiSmo dataset and the Corsican Fire Database based on various evaluation metrics. Comparative analysis shows that the proposed model outperforms previous fire segmentation deep learning models and image processing algorithms. [ABSTRACT FROM AUTHOR]

Purpose: Breast ultrasound (BUS) image segmentation plays a crucial role in computer‐aided diagnosis systems for BUS examination, which are useful for improved accuracy of breast cancer diagnosis. However, such performance remains a challenging task owing to the poor image quality and large variations in the sizes, shapes, and locations of breast lesions. In this paper, we propose a new convolutional neural network with coarse‐to‐fine feature fusion to address the aforementioned challenges. Methods: The proposed fusion network consists of an encoder path, a decoder path, and a core fusion stream path (FSP). The encoder path is used to capture the context information, and the decoder path is used for localization prediction. The FSP is designed to generate beneficial aggregate feature representations (i.e., various‐sized lesion features, aggregated coarse‐to‐fine information, and high‐resolution edge characteristics) from the encoder and decoder paths, which are eventually used for accurate breast lesion segmentation. To better retain the boundary information and alleviate the effect of image noise, we input the superpixel image along with the original image to the fusion network. Furthermore, a weighted‐balanced loss function was designed to address the problem of lesion regions having different sizes. We then conducted exhaustive experiments on three public BUS datasets to evaluate the proposed network. Results: The proposed method outperformed state‐of‐the‐art (SOTA) segmentation methods on the three public BUS datasets, with average dice similarity coefficients of 84.71(±1.07), 83.76(±0.83), and 86.52(±1.52), average intersection‐over‐union values of 76.34(±1.50), 75.70(±0.98), and 77.86(±2.07), average sensitivities of 86.66(±1.82), 85.21(±1.98), and 87.21(±2.51), average specificities of 97.92(±0.46), 98.57(±0.19), and 99.42(±0.21), and average accuracies of 95.89(±0.57), 97.17(±0.3), and 98.51(±0.3). Conclusions: The proposed fusion network could effectively segment lesions from BUS images, thereby presenting a new feature fusion strategy to handle challenging task of segmentation, while outperforming the SOTA segmentation methods. The code is publicly available at https://github.com/mniwk/CF2‐NET. [ABSTRACT FROM AUTHOR]

Recently, automatic diagnostic approaches have been widely used to classify ocular diseases. Most of these approaches are based on a single imaging modality (e.g., fundus photography or optical coherence tomography (OCT)), which usually only reflect the oculopathy to a certain extent, and neglect the modality-specific information among different imaging modalities. This paper proposes a novel modality-specific attention network (MSAN) for multi-modal retinal image classification, which can effectively utilize the modality-specific diagnostic features from fundus and OCT images. The MSAN comprises two attention modules to extract the modality-specific features from fundus and OCT images, respectively. Specifically, for the fundus image, ophthalmologists need to observe local and global pathologies at multiple scales (e.g., from microaneurysms at the micrometer level, optic disc at millimeter level to blood vessels through the whole eye). Therefore, we propose a multi-scale attention module to extract both the local and global features from fundus images. Moreover, large background regions exist in the OCT image, which is meaningless for diagnosis. Thus, a region-guided attention module is proposed to encode the retinal layer-related features and ignore the background in OCT images. Finally, we fuse the modality-specific features to form a multi-modal feature and train the multi-modal retinal image classification network. The fusion of modality-specific features allows the model to combine the advantages of fundus and OCT modality for a more accurate diagnosis. Experimental results on a clinically acquired multi-modal retinal image (fundus and OCT) dataset demonstrate that our MSAN outperforms other well-known single-modal and multi-modal retinal image classification methods. [ABSTRACT FROM AUTHOR]

The focus of this paper is to introduce an automated early Diabetic Retinopathy (DR) detection scheme from colour fundus images through enhanced segmentation and classification strategies by analyzing blood vessels. The occurrence of DR is increasing from the past years, impacting the eyes due to a sudden rise in the glucose level of blood. All over the world, half of the people who are under age 70 are severely suffered from diabetes. The patients who are affected by DR will lose their vision during the absence of early recognition of DR and appropriate treatment. To decrease the growth and occurrence of loss of vision, the early detection and timely treatment of DR are desirable. At present, deep learning models have presented better performance using retinal images for DR detection. In this work, the input retinal fundus images are initially subjected to pre-processing that undergoes contrast enhancement by Contrast Limited Adaptive Histogram Equalization (CLAHE) and average filtering. Further, the optimized binary thresholding-based segmentation is done for blood vessel segmentation. For the segmented image, Tri-level Discrete Level Decomposition (Tri-DWT) is performed to decompose it. In the feature extraction phase, Local Binary Pattern (LBP), and Gray-Level Co-occurrence Matrices (GLCMs) are extracted. Next, the classification of images is done through the combination of two algorithms, one is Neural Network (NN), and the other Convolutional Neural Network (CNN). The extracted features are subjected to NN, and the tri-DWT-based segmented image is subjected to CNN. Both the segmentation and classification phases are enhanced by the improved meta-heuristic algorithm called Fitness Rate-based Crow Search Algorithm (FR-CSA), in which few parameters are optimized for attaining maximum detection accuracy. The proposed DR detection model was implemented in MATLAB 2018a, and the analysis was done using three datasets, HRF, Messidor, and DIARETDB. The developed FR-CSA algorithm has the best detection accuracy in diagnosing DR. [ABSTRACT FROM AUTHOR]

Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more practical than full supervision for training segmentation algorithms. These methods have been predominantly developed to solve the background separation and partial segmentation problems presented by natural scene images and it is unclear whether they can be simply transferred to other domains with different characteristics, such as histopathology and satellite images, and still perform well. This paper evaluates state-of-the-art weakly-supervised semantic segmentation methods on natural scene, histopathology, and satellite image datasets and analyzes how to determine which method is most suitable for a given dataset. Our experiments indicate that histopathology and satellite images present a different set of problems for weakly-supervised semantic segmentation than natural scene images, such as ambiguous boundaries and class co-occurrence. Methods perform well for datasets they were developed on, but tend to perform poorly on other datasets. We present some practical techniques for these methods on unseen datasets and argue that more work is needed for a generalizable approach to weakly-supervised semantic segmentation. Our full code implementation is available on GitHub: https://github.com/lyndonchan/wsss-analysis. [ABSTRACT FROM AUTHOR]

• Cross-layer connected convolutional neural networks for 3D medical image segmentations. • Region of interest adaptive attention mechanism based on auxiliary loss function. • Accuracy improvement on livers, liver tumors, and brain tumors, especially for small tumors. 3D medical image segmentation is one of the most important and challenging tasks for medical image analysis. To enhance the feature extraction ability of 3D convolutional neural networks (CNN) and improve the segmentation accuracy on small tumors, this paper presents a cross-layer connected network with an adaptive attention mechanism to achieve efficient segmentations for 3D multi-organ and tumor from CT. A cross layer connection is proposed to utilize features more effectively by reusing multiple layers of semantic information while recovering spatial information. The pre-activation Residual Block and Squeeze-and-Excitation (SE) block are introduced to enhance the feature extraction ability. Due to the serious class imbalance problem caused by small tumors, the training process of the 3D CNN model is unstable and small targets are difficult to segment. To relieve the training instability problem, Anti-Aliased Pooling (AAP) is introduced to replace the original maxpooling. Moreover, a region of interest (ROI) adaptive attention mechanism based on the auxiliary loss function is developed to improve the segmentation accuracy on small target tumors by combining the target detection method with semantic segmentation models. Extensive experiments show that the dice coefficient of 3D liver tumor segmentation on the LiTS dataset reaches 0.668 by using our designed network, and the experiments on the BRATS dataset for 3D brain tumor segmentation yield average dice coefficient, sensitivity, and Hausdorff distance of 0.706, 0.730, and 5.972 mm, respectively, which are obviously superior to some existing methods. [ABSTRACT FROM AUTHOR]

Stroke, particularly ischemic stroke, is a major cause of disability and one of the leading causes of adult mortality worldwide. Early and prompt management of stroke patients can reduce the severity of the disease. Doctors usually determine the severity of a stroke by focusing on the region of interest (ROI) in the MRI or CT scan images. An accurate and effective automatic image segmentation system can assist medical professionals as well as automatic detection and classification systems. Deep learning is the current advanced approach for dealing with machine learning and artificial intelligence. However, conventional deep learning requires a large amount of data for training, and the amount of labeled data in the medical field is limited. In this paper, we propose a few-shot learning strategy and integrate it with a base convolutional neural network model, which utilizes a self-attention mechanism to segment MRI for ischemic stroke. By combining the base model with self-attention, we can focus more on the ROI and disregard less important features. Additionally, the proposed system only selects slices with lesions and ignores unlesioned slices. This helps to improve efficiency and reduce the computational load by eliminating the need to tune unnecessary parameters. To achieve even better results, the system also combines two weighted images, FLAIR and DWI, as an early fusion process. Experiments have shown that this approach leads to higher performance compared to using the same system without fusion. The proposed system is evaluated using a publicly available dataset, ISLES 2015 SSIS, and compared with other state-of-the-art (SOTA) systems. It achieves a dice coefficient score of 0.68, which is significantly better than that of other SOTA systems. • An automatic ischemic stroke segmentation system using few-shot learning is proposed. • The proposed system selects only slices with lesions and excludes unlesioned slices. • We adopted prototypical networks with metric learning to achieve few-shot segmentation. • Two weighted MRIs, FLAIR, and DWI are fused in the few-shot learning paradigm. [ABSTRACT FROM AUTHOR]