Search

The accurate representation of objects holds pivotal significance in the realm of panoptic segmentation. Presently, prevalent object representation methodologies, including box-based, keypoint-based and query-based techniques, encounter a challenge known as the 'representation confusion' issue in specific scenarios, often resulting in the mislabeling of instances. In response, this paper introduces Convex Object Representation (COR), a straightforward yet highly effective approach to address this problem. COR leverages a CNN-based Euclidean Distance Transform to convert the target instance into a convex heatmap. Simultaneously, it offers a parallel embedding method for encoding the object. Subsequently, COR characterizes objects based on the distinctive embedding vectors of their convex vertices. This paper seamlessly integrates COR into a state-of-the-art query-based panoptic segmentation framework. Experimental findings validate that COR successfully mitigates the representation confusion predicament, enhancing segmentation accuracy. The COR-augmented methods exhibit notable improvements of +1.3 and +0.7 points in PQ on the Cityscapes validation and MS COCO panoptic 2017 validation datasets, respectively. [ABSTRACT FROM AUTHOR]

Using instance segmentation and video inpainting provides a significant leap in real-time football video broadcast enhancements by removing potential visual distractions, such as an occasional person or another object accidentally occupying the frame. Despite its relevance and importance in the media industry, this area remains challenging and relatively understudied, thus offering potential for research. Specifically, the segmentation and inpainting of camera operator instances from video remains an underexplored research area. To address this challenge, this paper proposes a framework designed to accurately detect and remove camera operators while seamlessly hallucinating the background in real-time football broadcasts. The approach aims to enhance the quality of the broadcast by maintaining its consistency and level of engagement to retain and attract users during the game. To implement the inpainting task, firstly, the camera operators instance segmentation method should be developed. We used a YOLOv8 model for accurate real-time operator instance segmentation. The resulting model produces masked frames, which are used for further camera operator inpainting. Moreover, this paper presents an extensive "Cameramen Instances" dataset with more than 7500 samples, which serves as a solid foundation for future investigations in this area. The experimental results show that the YOLOv8 model performs better than other baseline algorithms in different scenarios. The precision of 95.5%, recall of 92.7%, mAP50-95 of 79.6, and a high FPS rate of 87 in low-volume environment prove the solution efficacy for real-time applications. [ABSTRACT FROM AUTHOR]

With the increasing complexity of the grid meter dial, precise feature extraction is becoming more and more difficult. Many automatic recognition solutions have been proposed for grid meter readings. However, traditional inspection methods cannot guarantee detection accuracy in complex environments. So, deep-learning methods are combined with grid meter recognition. Existing recognition systems that utilize segmentation models exhibit very high computation. It is challenging to ensure high real-time performance in edge computing devices. Therefore, an improved meter recognition model based on YOLOv7 is proposed in this paper. Partial convolution (PConv) is introduced into YOLOv7 to create a lighter network. Different PConv introduction locations on the base module have been used in order to find the optimal approach for reducing the parameters and floating point of operations (FLOPs). Meanwhile, the dynamic head (DyHead) module is utilized to enhance the attention mechanism for the YOLOv7 model. It can improve the detection accuracy of striped objects. As a result, this paper achieves mAP 50 v a l of 97.87% and mAP 50 : 90 v a l of 62.4% with only 5.37 M parameters. The improved model's inference speed can reach 108 frames per second (FPS). It enables detection accuracy that can reach ± 0.1 degrees in the grid meter. [ABSTRACT FROM AUTHOR]

Image semantic segmentation technology is one of the core research contents in the field of computer vision. With the improvement of computer performance and the continuous development of deep learning technology, researchers have more and more enthusiasm to study the actual effect and performance of image semantic segmentation. The results of deep semantic segmentation allow computers to have a more detailed and accurate understanding of images, and have a wide range of application needs in the fields of autonomous driving, intelligent security, medical imaging, remote sensing images, etc. However, the existing image semantic segmentation algorithms have the disadvantages of easy discontinuous results and insufficient prediction accuracy. In this paper, we take deep learning-based image semantic segmentation technology as the research object to explore the improvement of the image semantic segmentation algorithm and its application in road scenarios. First, this paper proposes MCU-Net method based on residual fusion and multi-scale contextual information. MCU-Net uses residual fusion module to deepen the network structure and improve the ability of U-Net to acquire deeper features. Then a top-down and bottom-up path is constructed for feature information between different levels, and the spatial and semantic information contained in shallow and deep features in the network is fully utilized by fusing features from different levels. In addition, an enhanced void space pyramid pooling module is added for feature information between the same levels, which enables the output features to have a larger range of semantic information. Second, this paper proposes the DAMCU-Net method based on attention mechanism and edge detection based on MCU-Net. DAMCU-Net extracts global contextual information by the attention mechanism optimization module, while fusing features using dense jump connections to facilitate the network to recover more spatial detail information during upsampling, and uses the FReLU activation function to improve the segmentation capability of the network for complex targets. For the edge information lost in the feature extraction process, the edge detection branch is added to supplement the feature information of the main path by feature fusion to achieve the optimization of the edge information. [ABSTRACT FROM AUTHOR]

Background Purpose Objective Methods Results Conclusions Cancer, a disease with a high mortality rate, poses a great threat to patients' physical and mental health and can lead to huge medical costs and emotional damage. With the continuous development of artificial intelligence technologies, deep learning‐based cancer image segmentation techniques are becoming increasingly important in cancer detection and accurate diagnosis. However, in segmentation tasks, there are differences in efficiency between large and small objects and limited segmentation effects on objects of individual sizes. The previous segmentation frameworks still have room for improvement in multi‐scale collaboration when segmenting objects.This paper proposes a method to train a deep learning segmentation framework using a feature pyramid processing dataset to improve the average precision (AP) index, and realizes multi‐scale cooperation in target segmentation.Pan‐Cancer Histology Dataset for Nuclei Instance Segmentation and Classification (PanNuke) dataset was selected to include approximately 7500 pathology images with cells from 19 different types of tissues, including five classifications of cancer, non‐cancer, inflammation, death, and connective tissue.First, the method uses whole‐slide images in the pan‐cancer histology dataset for nuclei instance segmentation and classification (PanNuke) dataset, combined with the mask region convolutional neural network (Mask R‐CNN) segmentation framework and improved loss function to segment and detect each cellular tissue in cancerous sections. Second, to address the problem of non‐synergistic object segmentation at different scales in cancerous tissue segmentation, a scheme using feature pyramids to process the dataset was adopted as part of the feature extraction module.Extensive experimental results on this dataset show that the method in this paper yields 0.269 AP and a boost of about 4% compared to the original Mask R‐CNN framework.It is effective and feasible to use feature pyramid to process data set to improve the effect of medical image segmentation. [ABSTRACT FROM AUTHOR]

Microscopic image analysis technology helps solve the inadvertences of artificial traditional methods in disease, wastewater treatment, and environmental change monitoring analysis. Convolutional neural network (CNN) play an important role in microscopic image analysis. Image segmentation, in which U-Net is increasingly applied in microscopic image segmentation, is a crucial step in detection, tracking, monitoring, feature extraction, modelling, and analysis. This paper comprehensively reviews the development history of U-Net, analyses several research results of various segmentation methods since the emergence of U-Net, and conducts a comprehensive review of related papers. This paper summarised the improved methods of U-Net and then listed the existing significance of image segmentation techniques and their improvements introduced over the years. Finally, focusing on the different improvement strategies of U-Net in different papers, the related work of each application target is reviewed according to detailed technical categories to facilitate future research. Researchers can see the dynamics of the transmission of technological development and keep up with future trends in this interdisciplinary field. [ABSTRACT FROM AUTHOR]

Blinding eye diseases are often related to changes in retinal structure, which can be detected by analysing retinal blood vessels in fundus images. However, existing techniques struggle to accurately segment these delicate vessels. Although deep learning has shown promise in medical image segmentation, its reliance on specific operations can limit its ability to capture crucial details such as the edges of the vessel. This paper introduces LMBiS-Net, a lightweight convolutional neural network designed for the segmentation of retinal vessels. LMBiS-Net achieves exceptional performance with a remarkably low number of learnable parameters (only 0.172 million). The network used multipath feature extraction blocks and incorporates bidirectional skip connections for the information flow between the encoder and decoder. In addition, we have optimised the efficiency of the model by carefully selecting the number of filters to avoid filter overlap. This optimisation significantly reduces training time and improves computational efficiency. To assess LMBiS-Net's robustness and ability to generalise to unseen data, we conducted comprehensive evaluations on four publicly available datasets: DRIVE, STARE, CHASE_DB1, and HRF The proposed LMBiS-Net achieves significant performance metrics in various datasets. It obtains sensitivity values of 83.60%, 84.37%, 86.05%, and 83.48%, specificity values of 98.83%, 98.77%, 98.96%, and 98.77%, accuracy (acc) scores of 97.08%, 97.69%, 97.75%, and 96.90%, and AUC values of 98.80%, 98.82%, 98.71%, and 88.77% on the DRIVE, STARE, CHEASE_DB, and HRF datasets, respectively. In addition, it records F1 scores of 83.43%, 84.44%, 83.54%, and 78.73% on the same datasets. Our evaluations demonstrate that LMBiS-Net achieves high segmentation accuracy (acc) while exhibiting both robustness and generalisability across various retinal image datasets. This combination of qualities makes LMBiS-Net a promising tool for various clinical applications. [ABSTRACT FROM AUTHOR]

The accurate and automated segmentation of neurons in electron microscopy (EM) images presents a challenging problem in neuroscience. In this paper, we propose a deep learning (DL)-based neuron segmentation approach using a relatively shallow residual deconvolutional neural network (DNN) as the core. Our proposed segmentation framework differs from most EM image segmentation networks that use original images as input, as we suggest using modified morphological features with attention characteristics as input instead. We calculate Bhattacharyya distances in morphology feature space to provide attention features, which are then concatenated with the original image and used as input to the segmentation network. By attending to membrane pixels, our network efficiently highlights the discrimination between membrane boundaries and background pixels. To remove noise pixels in the segmentation map while preserving membrane boundaries in the final result, the dilated version of the output is then processed using a guided filtering scenario. Our proposed model outperforms several competing DL methods such as DeepLabv3 as well as different versions of deconvolutional neural networks on the ISBI 2012 dataset in terms of various image segmentation metrics. The proposed attention-based morphological guided framework can help advance research in neuroscience by providing an automated and accurate segmentation of neurons in EM images. [ABSTRACT FROM AUTHOR]

High‐fidelity segmentation of blood vessels plays a pivotal role in numerous biomedical applications, such as injection assistance, cancer detection, various surgeries, and vein authentication. Near‐infrared (NIR) transillumination imaging is an effective and safe method to visualize the subcutaneous blood vessel network. However, such images are severely blurred because of the light scattering in body tissues. Inspired by the Vision Transformer model, this paper proposes a novel deep learning network known as transformer connection (TRC)‐Unet to capture global blurred and local clear correlations while using multi‐layer attention. Our method mainly consists of two blocks, thereby aiming to remap skip connection information flow and fuse different domain features. Specifically, the TRC extracts global blurred information from multiple layers and suppresses scattering to increase the clarity of vessel features. Transformer feature fusion eliminates the domain gap between the highly semantic feature maps of the convolutional neural network backbone and the adaptive self‐attention maps of TRCs. Benefiting from the long‐range dependencies of transformers, we achieved competitive results in relation to various competing methods on different data sets, including retinal vessel segmentation, simulated blur image segmentation, and real NIR blood vessel image segmentation. Moreover, our method remarkably improved the segmentation results of simulated blur image data sets and a real NIR vessel image data set. The quantitative results of ablation studies and visualizations are also reported to demonstrate the superiority of the TRC‐Unet design. © 2024 The Author(s). IEEJ Transactions on Electrical and Electronic Engineering published by Institute of Electrical Engineers of Japan and Wiley Periodicals LLC. [ABSTRACT FROM AUTHOR]

Deep learning has attained state-of-the-art results in general image segmentation problems; however, it requires a substantial number of annotated images to achieve the desired outcomes. In the medical field, the availability of annotated images is often limited. To address this challenge, few-shot learning techniques have been successfully adapted to rapidly generalize to new tasks with only a few samples, leveraging prior knowledge. In this paper, we employ a gradient-based method known as Model-Agnostic Meta-Learning (MAML) for medical image segmentation. MAML is a meta-learning algorithm that quickly adapts to new tasks by updating a model's parameters based on a limited set of training samples. Additionally, we use an enhanced 3D U-Net as the foundational network for our models. The enhanced 3D U-Net is a convolutional neural network specifically designed for medical image segmentation. We evaluate our approach on the TotalSegmentator dataset, considering a few annotated images for four tasks: liver, spleen, right kidney, and left kidney. The results demonstrate that our approach facilitates rapid adaptation to new tasks using only a few annotated images. In 10-shot settings, our approach achieved mean dice coefficients of 93.70%, 85.98%, 81.20%, and 89.58% for liver, spleen, right kidney, and left kidney segmentation, respectively. In five-shot sittings, the approach attained mean Dice coefficients of 90.27%, 83.89%, 77.53%, and 87.01% for liver, spleen, right kidney, and left kidney segmentation, respectively. Finally, we assess the effectiveness of our proposed approach on a dataset collected from a local hospital. Employing five-shot sittings, we achieve mean Dice coefficients of 90.62%, 79.86%, 79.87%, and 78.21% for liver, spleen, right kidney, and left kidney segmentation, respectively. [ABSTRACT FROM AUTHOR]

Breast tumor segmentation of ultrasound images provides valuable tumor information for early detection and diagnosis. However, speckle noise and blurred boundaries in breast ultrasound images present challenges for tumor segmentation, especially for malignant tumors with irregular shapes. Recent vision transformers have shown promising performance in handling the variation through global context modeling. Nevertheless, they are often dominated by features of large patterns and lack the ability to recognize negative information in ultrasound images, which leads to the loss of breast tumor details (e.g., boundaries and small objects). In this paper, we propose a novel noise suppression and boundary refinement network, NSBR-Net, to simultaneously alleviate speckle noise interference and blurred boundary problems of breast tumor segmentation. Specifically, we propose two innovative designs, namely, the Noise Suppression Module (NSM) and the Boundary Refinement Module (BRM). The NSM filters noise information from the coarse-grained feature maps, while the BRM progressively refines the boundaries of significant lesion objects. Our method demonstrates superior accuracy over state-of-the-art deep learning models, achieving significant improvements of 3.67% on Dataset B and 2.30% on the BUSI dataset in mDice for testing malignant tumors. [ABSTRACT FROM AUTHOR]

Weeds pose a significant threat to agricultural production, leading to substantial yield losses and increased herbicide usage, with severe economic and environmental implications. This paper uses deep learning to explore a novel approach via targeted segmentation mapping of crop plants rather than weeds, focusing on canola (Brassica napus) as the target crop. Multiple deep learning architectures (ResNet-18, ResNet-34, and VGG-16) were trained for the pixel-wise segmentation of canola plants in the presence of other plant species, assuming all non-canola plants are weeds. Three distinct datasets (T1_miling, T2_miling, and YC) containing 3799 images of canola plants in varying field conditions alongside other plant species were collected with handheld devices at 1.5 m. The top performing model, ResNet-34, achieved an average precision of 0.84, a recall of 0.87, a Jaccard index (IoU) of 0.77, and a Macro F1 score of 0.85, with some variations between datasets. This approach offers increased feature variety for model learning, making it applicable to the identification of a wide range of weed species growing among canola plants, without the need for separate weed datasets. Furthermore, it highlights the importance of accounting for the growth stage and positioning of plants in field conditions when developing weed detection models. The study contributes to the growing field of precision agriculture and offers a promising alternative strategy for weed detection in diverse field environments, with implications for the development of innovative weed control techniques. [ABSTRACT FROM AUTHOR]

This paper delves into the methodology employed in examining lean premixed turbulent flame fronts extracted from Planar Laser Induced Fluorescence (PLIF) images at elevated pressures. In such flow regimes, the PLIF signal suffers from significant collisional quenching, typically resulting in image data with low signal-to-noise ratio (SNR). This poses severe difficulties for conventional flame front extraction algorithms based on intensity gradients and requires intense user intervention to yield acceptable results. In this work, we propose Convolutional Neural Network (CNN)-based Deep Learning (DL) models as an alternative to problem specific conventional methods. The pretrained DL models were fine-tuned, employing data augmentation, on a small annotated dataset including a variety of conditions between SNR ≈ 1.6 to 2.6 and subsequently evaluated. All DL models significantly outperformed the best-scoring conventional implementation both quantitatively and visually, while having similar inference times. IoU-scores and Recall values were found to be up to a factor ≈ 1.2 and ≈ 2.5 higher, respectively, with ≈ 1.15 times improved Precision. Small-scale structures were captured much better with fewer erroneous predictions, becoming particularly pronounced for the lower SNR data investigated. Moreover, by applying artificially modeled noise, it was shown that the range of image conditions in terms of SNR that can be reliably processed extends well beyond the images included in the training data, and satisfactory segmentation performances were found for SNR as low as ≈ 1.1. The presented DL-based flame front detection algorithm marks a methodology with significantly increased detection performance, while a similar computational effort for inference is achieved and the need for user-based parameter tuning is eliminated. It enables a very accurate extraction of instantaneous flame fronts in large image datasets where supervised processing is infeasible, unlocking unprecedented possibilities for the study of flame dynamics and instability mechanisms at industry-relevant conditions. [ABSTRACT FROM AUTHOR]

Structural health monitoring for roads is an important task that supports inspection of transportation infrastructure. This paper explores deep learning techniques for crack detection in road images and proposes an automatic pixel-level semantic road crack image segmentation method based on a Swin transformer. This method employs Swin-T as the backbone network to extract feature information from crack images at various levels and utilizes the texture unit to extract the texture and edge characteristic information of cracks. The refinement attention module (RAM) and panoramic feature module (PFM) then merge these diverse features, ultimately refining the segmentation results. This method is called FetNet. We collect four public real-world datasets and conduct extensive experiments, comparing FetNet with various deep-learning methods. FetNet achieves the highest precision of 90.4%, a recall of 85.3%, an F1 score of 87.9%, and a mean intersection over union of 78.6% on the Crack500 dataset. The experimental results show that the FetNet approach surpasses other advanced models in terms of crack segmentation accuracy and exhibits excellent generalizability for use in complex scenes. [ABSTRACT FROM AUTHOR]

The efficient and timely identification of oil spill areas is crucial for ocean environmental protection. Synthetic aperture radar (SAR) is widely used in oil spill detection due to its all-weather monitoring capability. Meanwhile, existing deep learning-based oil spill detection methods mainly rely on the classical U-Net framework and have achieved impressive results. However, SAR images exhibit high noise, blurry boundaries, and irregular shapes of target areas, as well as speckles and shadows, which lead to the loss of performance in existing algorithms. In this paper, we propose a novel network architecture to achieve more precise segmentation of oil spill areas by reintroducing rich semantic contextual information before obtaining the final segmentation mask. Specifically, the proposed architecture can re-fuse feature maps from different levels at the decoder end. We design a multi-convolutional layer (MCL) module to extract basic feature information from SAR images, and a feature extraction module (FEM) module further extracts and fuses feature maps generated by the U-Net decoder at different levels. Through these operations, the network can learn rich global and local contextual information, enable sufficient interaction of feature information at different stages, enhance the model's contextual awareness, and improve its ability to recognize complex textures and blurry boundaries, thereby enhancing the segmentation accuracy of SAR images. Compared to many U-Net based segmentation networks, our method shows promising results and achieves state-of-the-art performance on multiple evaluation metrics. [ABSTRACT FROM AUTHOR]

The use of deep learning-based high resolution remote sensing image sea and land segmentation method has become prevalent in various fields such as environmental monitoring, resource assessment, and urban planning. However, the segmentation process faces challenges due to the complex coastline and the similarity in spectral colour difference between the sea and land in certain areas. The remote sensing images obtained from satellites often exhibit unclear boundaries between the sea and land, as well as intricate coastlines. Consequently, the segmentation of sea and land becomes a difficult task. The existing sea-land segmentation methods have limitations in accurately segmenting long and narrow waters, as well as areas with low differentiation between sea and land. In this paper, we propose a new sea-land segmentation method called SRMA (Swin-Res-Multi-Attention). SRMA performs parallel feature extraction using a dual-branch parallel: Swin-Transformer branch and ResNet branch. The combination of Swin-Transformer's self-attention and ResNet's residual design helps reduce the loss of feature information, thereby avoiding incorrect segmentation in regions with low distinctness. To improve overall stability, we introduce the MCA (Multiscale channel attention module) module, which combines feature information from different scales. The multi-scale pyramid design of MCA module ensures stability and improves segmentation accuracy in narrow waters. Experimental results on a public dataset demonstrate that our model outperforms existing methods. Additionally, we evaluate the model's performance in complex scenarios by creating a new dataset that includes a complex urban water body and cloud interference. The results showcase the robustness and superiority of our model. [ABSTRACT FROM AUTHOR]

The paper explores the tensile behaviour of plain weave ceramic matrix composites (CMCs) by employing a nonlinear constitutive model for fibre tow composites, driven by in-situ X-ray computed tomography data. Through the application of a deep-learning-based image segmentation method, the study successfully identified material damage modes, including matrix cracks, fibre pull-out, and interface debonding. Three-dimensional characterisation and quantitative analyses of damage have been conducted. At the mesoscopic level, a theoretically derived nonlinear constitutive model for fibre tow composites, accounting for continuous damage to both fibre and matrix, is established by using damage data. At the macroscopic level, a unit cell model is constructed for the numerical simulation of plain weave CMCs. Material's initial defects are prone to induce crack initiation. External matrix cracks can propagate to tows, and crack deflection may occur during this process. Furthermore, adjacent microcracks have the potential to connect, forming a larger one under high stress. Theoretical results predicted by the nonlinear constitutive model for fibre tow composites under tension exhibit good agreement with test data. The built unit cell model accurately predicts the tensile mechanical behaviour of plaine weave CMCs. [ABSTRACT FROM AUTHOR]

BACKGROUND: Cardiovascular diseases are the top cause of death in China. Manual segmentation of cardiovascular images, prone to errors, demands an automated, rapid, and precise solution for clinical diagnosis. OBJECTIVE: The paper highlights deep learning in automatic cardiovascular image segmentation, efficiently identifying pixel regions of interest for auxiliary diagnosis and research in cardiovascular diseases. METHODS: In our study, we introduce innovative Region Weighted Fusion (RWF) and Shape Feature Refinement (SFR) modules, utilizing polarized self-attention for significant performance improvement in multiscale feature integration and shape fine-tuning. The RWF module includes reshaping, weight computation, and feature fusion, enhancing high-resolution attention computation and reducing information loss. Model optimization through loss functions offers a more reliable solution for cardiovascular medical image processing. RESULTS: Our method excels in segmentation accuracy, emphasizing the vital role of the RWF module. It demonstrates outstanding performance in cardiovascular image segmentation, potentially raising clinical practice standards. CONCLUSIONS: Our method ensures reliable medical image processing, guiding cardiovascular segmentation for future advancements in practical healthcare and contributing scientifically to enhanced disease diagnosis and treatment. [ABSTRACT FROM AUTHOR]

Recent studies show that neural networks are vulnerable to backdoor attacks, in which compromised networks behave normally for clean inputs but make mistakes when a pre-defined trigger appears. Although prior studies have designed various invisible triggers to avoid causing visual anomalies, they cannot evade some trigger detectors. In this paper, we consider the stealthiness of backdoor attacks from input space and feature representation space. We propose a novel backdoor attack named sparse backdoor attack, and investigate the minimum required trigger to induce the well-trained networks to make incorrect results. A U-net-based generator is employed to create triggers for each clean image. Considering the stealthiness of the trigger, we restrict the elements of the trigger between −1 and 1. In the aspect of the feature representation domain, we adopt an entanglement cost function to minimize the gap between feature representations of benign and malicious inputs. The inseparability of benign and malicious feature representations contributes to the stealthiness of our attack against various model diagnosis-based defences. We validate the effectiveness and generalization of our method by conducting extensive experiments on multiple datasets and networks. [ABSTRACT FROM AUTHOR]

Unmanned Aerial Vehicle (UAV) has seen a dramatic rise in popularity for remote-sensing image acquisition and analysis in recent years. It has brought promising results in low-altitude monitoring tasks that require detailed visual inspections. Semantic segmentation is one of the hot topics in UAV remote sensing image analysis, as its capability to mine contextual semantic information from UAV images is crucial for achieving a fine-grained understanding of scenes. However, in the remote sensing field, recent reviews have not focused on combining "UAV remote sensing" and "semantic segmentation" to summarize the advanced works and future trends. In this study, we focus primarily on describing various recent semantic segmentation methods applied in UAV remote sensing images and summarizing their advantages and limitations. According to the distinction in modeling contextual semantic information, we have categorized and outlined the methods based on graph-based contextual models and deep-learning-based models. Publicly available UAV-based image datasets are also gathered to encourage systematic research on advanced semantic segmentation methods. We provide quantitative results of representative methods on two high-resolution UAV-based image datasets for fair comparisons and discussions in terms of semantic segmentation accuracy and model inference efficiency. Besides, this paper concludes some remaining challenges and future directions in semantic segmentation for UAV remote sensing images and points out that methods based on deep learning will become the future research trend. • Summarize semantic segmentation methods and datasets for UAV remote sensing images. • Provide fair comparisons of representative methods in accuracy and efficiency. • Identify challenges and future directions for UAV image semantic segmentation. • Semantic segmentation methods based on deep learning will be the mainstream. [ABSTRACT FROM AUTHOR]

Anemia is a severe health condition commonly prevalent among women of reproductive age and children below five years. Screening patients before the condition becomes critical and can save many lives. World Health Organization (WHO) has set the "Global nutrition target 2025-anemia," aiming to reduce 50% of anemia cases among women of reproductive age. This target can be achieved through a time-efficient, cost-effective, and easy-to-use tool. Traditional testing methods require specific chemicals, machines, and equipment that are not available everywhere. It also requires the presence of nurses, laboratory workers, and doctors. These methods are costly, time-consuming, and produce biohazard waste, thus polluting the environment. We developed an Artificial Intelligence (AI)-based bot that can be used for screening people for anemia. The bot service is based on two models: a segmentation model to segment the Region of Interest (ROI) and a classification model to classify anemic cases from normal ones. To train the model, we have collected data from 160 anemic and 140 non-anemic persons. In this paper, we have explained the architecture of the models, all the training parameters, and their deployment on cloud services using the REAN chatbot service. We manage to reach an Intersection Over Union (IOU) score of 0.922 for the segmentation model; validation recall of 0.95 and validation accuracy of 0.9699 for the classification model. This system is easy to use and does not depend on the availability of comprehensive laboratory infrastructure or trained personnel and thus can enable screening of anemia in low-resource settings. [ABSTRACT FROM AUTHOR]

Medical image segmentation is crucial for clinical applications, but challenges persist due to noise and variability. In particular, accurate glottis segmentation from high-speed videos is vital for voice research and diagnostics. Manual searching for failed segmentations is labor-intensive, prompting interest in automated methods. This paper proposes the first deep learning approach for detecting faulty glottis segmentations. For this purpose, faulty segmentations are generated by applying both a poorly performing neural network and perturbation procedures to three public datasets. Heavy data augmentations are added to the input until the neural network's performance decreases to the desired mean intersection over union (IoU). Likewise, the perturbation procedure involves a series of image transformations to the original ground truth segmentations in a randomized manner. These data are then used to train a ResNet18 neural network with custom loss functions to predict the IoU scores of faulty segmentations. This value is then thresholded with a fixed IoU of 0.6 for classification, thereby achieving 88.27% classification accuracy with 91.54% specificity. Experimental results demonstrate the effectiveness of the presented approach. Contributions include: (i) a knowledge-driven perturbation procedure, (ii) a deep learning framework for scoring and detecting faulty glottis segmentations, and (iii) an evaluation of custom loss functions. [ABSTRACT FROM AUTHOR]

In recent years, several studies have undergone automatic blood vessel segmentation based on unsupervised and supervised algorithms to reduce user interruption. Deep learning networks have been used to get highly accurate segmentation results. However, the incorrect segmentation of pathological information and low micro-vascular segmentation is considered the challenges present in the existing methods for segmenting the retinal blood vessel. It also affects different degrees of vessel thickness, contextual feature fusion in technique, and perception of details. A deep learning-aided method has been presented to address these challenges in this paper. In the first phase, the preprocessing is performed using the retinal fundus images employed by the black ring removal, LAB conversion, CLAHE-based contrast enhancement, and grayscale image. Thus, the blood vessel segmentation is performed by a new deep learning model termed optimized ResUNet + +. As an improvement to this deep learning architecture, the activation function is optimized by the J-AGSO algorithm. The objective function for the optimized ResUNet + + -based blood vessel segmentation is to minimize the binary cross-entropy loss function. Further, the post-processing of the images is carried out by median filtering and binary thresholding. By verifying the standard benchmark datasets, the proposed model outperforms and attains enhanced performance. [ABSTRACT FROM AUTHOR]

This paper designed and developed an online digital imaging excitation sensor for wind power gearbox wear condition monitoring based on an adaptive deep learning method. A digital imaging excitation sensing image information collection architecture for magnetic particles in lubricating oil was established to characterize the wear condition of mechanical equipment, achieving the real-time online collection of wear particles in lubricating oil. On this basis, a mechanical equipment wear condition diagnosis method based on online wear particle images is proposed, obtaining data from an engineering test platform based on a wind power gearbox. Firstly, a foreground segmentation preprocessing method based on the U-Net network can effectively eliminate the interference of bubbles and dark fields in online wear particle images, providing high-quality segmentation results for subsequent image processing, A total of 1960 wear particle images were collected in the experiment, the average intersection union ratio of the validation set is 0.9299, and the accuracy of the validation set is 0.9799. Secondly, based on the foreground segmentation preprocessing of wear particle images, by using the watered algorithm to obtain the number of particles in each size segment, we obtained the number of magnetic particle grades in three different ranges: 4–38 µm, 39–70 µm, and >70 µm. Thirdly, we proposed a method named multidimensional transformer (MTF) network. Mean Square Error (MSE), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE) are used to obtain the error, and the maintenance strategy is formulated according to the predicted trend. The experimental results show that the predictive performance of our proposed model is better than that of LSTM and TCN. Finally, the online real-time monitoring system triggered three alarms, and at the same time, our offline sampling data analysis was conducted, the accuracy of online real-time monitoring alarms was verified, and the gearbox of the wind turbine was shut down for maintenance and repair. [ABSTRACT FROM AUTHOR]

Automatic and accurate segmentation of medical images plays an essential role in disease diagnosis and treatment planning. Convolution neural networks have achieved remarkable results in medical image segmentation in the past decade. Meanwhile, deep learning models based on Transformer architecture also succeeded tremendously in this domain. However, due to the ambiguity of the medical image boundary and the high complexity of physical organization structures, implementing effective structure extraction and accurate segmentation remains a problem requiring a solution. In this paper, we propose a novel Dual Encoder Network named DECTNet to alleviate this problem. Specifically, the DECTNet embraces four components, which are a convolution-based encoder, a Transformer-based encoder, a feature fusion decoder, and a deep supervision module. The convolutional structure encoder can extract fine spatial contextual details in images. Meanwhile, the Transformer structure encoder is designed using a hierarchical Swin Transformer architecture to model global contextual information. The novel feature fusion decoder integrates the multi-scale representation from two encoders and selects features that focus on segmentation tasks by channel attention mechanism. Further, a deep supervision module is used to accelerate the convergence of the proposed method. Extensive experiments demonstrate that, compared to the other seven models, the proposed method achieves state-of-the-art results on four segmentation tasks: skin lesion segmentation, polyp segmentation, Covid-19 lesion segmentation, and MRI cardiac segmentation. [ABSTRACT FROM AUTHOR]

Image segmentation is critical in medical image processing for lesion detection, localisation, and subsequent diagnosis. Currently, computer-aided diagnosis (CAD) has played a significant role in improving diagnostic efficiency and accuracy. The segmentation task is made more difficult by the hazy lesion boundaries and uneven forms. Because standard convolutional neural networks (CNNs) are incapable of capturing global contextual information, adequate segmentation results are impossible to achieve. We propose a multiscale feature fusion network (MTC-Net) in this paper that integrates deep separable convolution and self-attentive modules in the encoder to achieve better local continuity of images and feature maps. In the decoder, a multi-branch multi-scale feature fusion module (MSFB) is utilized to improve the network's feature extraction capability, and it is integrated with a global cooperative aggregation module (GCAM) to learn more contextual information and adaptively fuse multi-scale features. To develop rich hierarchical representations of irregular forms, the suggested detail enhancement module (DEM) adaptively integrates local characteristics with their global dependencies. To validate the effectiveness of the proposed network, we conducted extensive experiments, evaluated on the public datasets of skin, breast, thyroid and gastrointestinal tract with ISIC2018, BUSI, TN3K and Kvasir-SEG. The comparison with the latest methods also verifies the superiority of our proposed MTC-Net in terms of accuracy. Our code on https://github.com/gih23/MTC-Net. [ABSTRACT FROM AUTHOR]

The segmentation of neuronal morphology in electron microscopy images is crucial for the analysis and understanding of neuronal function. However, most of the existing segmentation methods are not suitable for challenging datasets where the neuronal structure is contaminated by noise or has interrupted parts. In this paper, we propose a segmentation method based on deep learning to determine the location information of neurons and reduce the influence of image noise in the data. Specifically, we adapt our neuron dataset based on UNet by using convolution with BN fusion and multi-input feature fusion. The method is named REDAFNet. The model simplifies the model structure and enhances the generalization ability by fusing the convolution layer and BN layer. The noise interference in the data was reduced by multi-input feature fusion, and the ability to understand and express the data was enhanced. The method takes a neuron image as input and its pixel segmentation map as output. Experimental results show that the segmentation accuracy of the proposed method is 91.96%, 93.86% and 80.25% on the ISBI2012 dataset, U-RISC retinal neuron dataset and N2DH-GOWT1 stem cell dataset, respectively. Compared with the existing segmentation methods, the proposed method can extract more complete feature information and achieve more accurate segmentation. [ABSTRACT FROM AUTHOR]

Test-time adaptation (TTA) has been proven to effectively improve the adaptability of deep learning semantic segmentation models facing continuous changeable scenes. However, most of the existing TTA algorithms lack an explicit exploration of domain gaps, especially those based on visual domain prompts. To address these issues, this paper proposes a self-prompt strategy based on uncertainty, guiding the model to continuously focus on regions with high uncertainty (i.e., regions with a larger domain gap). Specifically, we still use the Mean-Teacher architecture with the predicted entropy from the teacher network serving as the input to the prompt module. The prompt module processes uncertain maps and guides the student network to focus on regions with higher entropy, enabling continuous adaptation to new scenes. This is a self-prompting strategy that requires no prior knowledge and is tested on widely used benchmarks. In terms of the average performance, our method outperformed the baseline algorithm in TTA and continual TTA settings of Cityscapes-to-ACDC by 3.3% and 3.9%, respectively. Our method also outperformed the baseline algorithm by 4.1% and 3.1% on the more difficult Cityscapes-to-(Foggy and Rainy) Cityscapes setting, which also surpasses six other current TTA methods. [ABSTRACT FROM AUTHOR]

The early diagnosis of diabetic retinopathy (DR) can effectively prevent irreversible vision loss and assist ophthalmologists in providing timely and accurate treatment plans. However, the existing methods based on deep learning have a weak perception ability of different scale information in retinal fundus images, and the segmentation capability of subtle lesions is also insufficient. This paper aims to address these issues and proposes MLNet for DR lesion segmentation, which mainly consists of the Multi-Scale Attention Block (MSAB) and the Lesion Perception Block (LPB). The MSAB is designed to capture multi-scale lesion features in fundus images, while the LPB perceives subtle lesions in depth. In addition, a novel loss function with tailored lesion weight is designed to reduce the influence of imbalanced datasets on the algorithm. The performance comparison between MLNet and other state-of-the-art methods is carried out in the DDR dataset and DIARETDB1 dataset, and MLNet achieves the best results of 51.81% mAUPR, 49.85% mDice, and 37.19% mIoU in the DDR dataset, and 67.16% mAUPR and 61.82% mDice in the DIARETDB1 dataset. The generalization experiment of MLNet in the IDRiD dataset achieves 59.54% mAUPR, which is the best among other methods. The results show that MLNet has outstanding DR lesion segmentation ability. [ABSTRACT FROM AUTHOR]

The precise classification of kidneys and kidney tumors is of great guiding significance for the diagnosis and surgical planning of clinical doctors. Artificial segmentation of kidneys and renal tumors not only increases the workload of doctors, but also has a certain degree of subjectivity. Due to the varying morphology and pathological characteristics of various kidney tumors, conventional imaging analysis methods have certain limitations in their application. This study aims to propose a novel optical imaging device based on deep learning to achieve better results in kidney tumor image segmentation and recognition. The study collected a batch of image data on kidney tumors and trained the data using deep learning algorithms. After many experiments and optimization, a model with excellent performance in renal tumor segmentation and recognition was obtained. Compared with several existing methods, the method proposed in this paper has better segmentation accuracy for kidney tumors. This can not only avoid mutual interference between the network and the resolution network, but also improve the recognition efficiency between the two networks. In addition, the system can also provide reliable data feedback for clinical doctors, helping them continuously correct problems in subsequent visual examinations, thereby achieving higher diagnostic accuracy. The experimental results show that the neural network is effective in image segmentation and quantitative analysis, and can be applied in clinical practice. [ABSTRACT FROM AUTHOR]

Accurate urban green space (UGS) measurement has become crucial for landscape analysis. This paper reviews the recent technological breakthroughs in deep learning (DL)-based semantic segmentation, emphasizing efficient landscape analysis, and integrating greenness measurements. It explores quantitative greenness measures applied through semantic segmentation, categorized into the plan view- and the perspective view-based methods, like the Land Class Classification (LCC) with green objects and the Green View Index (GVI) based on street photographs. This review navigates from traditional to modern DL-based semantic segmentation models, illuminating the evolution of the urban greenness measures and segmentation tasks for advanced landscape analysis. It also presents the typical performance metrics and explores public datasets for constructing these measures. The results show that accurate (semantic) segmentation is inevitable not only for fine-grained greenness measures but also for the qualitative evaluation of landscape analyses for planning amidst the incomplete explainability of the DL model. Also, the unsupervised domain adaptation (UDA) in aerial images is addressed to overcome the scale changes and lack of labeled data for fine-grained greenness measures. This review contributes to helping researchers understand the recent breakthroughs in DL-based segmentation technology for challenging topics in UGS research. [ABSTRACT FROM AUTHOR]

Polyp detection is a challenging task in the diagnosis of Colorectal Cancer (CRC), and it demands clinical expertise due to the diverse nature of polyps. The recent years have witnessed the development of automated polyp detection systems to assist the experts in early diagnosis, considerably reducing the time consumption and diagnostic errors. In automated CRC diagnosis, polyp segmentation is an important step which is carried out with deep learning segmentation models. Recently, Vision Transformers (ViT) are slowly replacing these models due to their ability to capture long range dependencies among image patches. However, the existing ViTs for polyp do not harness the inherent self-attention abilities and incorporate complex attention mechanisms. This paper presents Polyp-Vision Transformer (Polyp-ViT), a novel Transformer model based on the conventional Transformer architecture, which is enhanced with adaptive mechanisms for feature extraction and positional embedding. Polyp-ViT is tested on the Kvasir-seg and CVC-Clinic DB Datasets achieving segmentation accuracies of 0.9891 ± 0.01 and 0.9875 ± 0.71 respectively, outperforming state-of-the-art models. Polyp-ViT is a prospective tool for polyp segmentation which can be adapted to other medical image segmentation tasks as well due to its ability to generalize well. [ABSTRACT FROM AUTHOR]

Given that defect detection in weld X-ray images is a critical aspect of pressure vessel manufacturing and inspection, accurate differentiation of the type, distribution, number, and area of defects in the images serves as the foundation for judging weld quality, and the segmentation method of defects in digital X-ray images is the core technology for differentiating defects. Based on the publicly available weld seam dataset GDX-ray, this paper proposes a complete technique for fault segmentation in X-ray pictures of pressure vessel welds. The key works are as follows: (1) To address the problem of a lack of defect samples and imbalanced distribution inside GDX-ray, a DA-DCGAN based on a two-channel attention mechanism is devised to increase sample data. (2) A convolutional block attention mechanism is incorporated into the coding layer to boost the accuracy of small-scale defect identification. The proposed MAU-Net defect semantic segmentation network uses multi-scale even convolution to enhance large-scale features. The proposed method can mask electrostatic interference and non-defect-class parts in the actual weld X-ray images, achieve an average segmentation accuracy of 84.75% for the GDX-ray dataset, segment and accurately rate the valid defects with a correct rating rate of 95%, and thus realize practical value in engineering. [ABSTRACT FROM AUTHOR]

Contrails are line-shaped clouds formed in the exhaust of aircraft engines that significantly contribute to global warming. This paper confidently proposes integrating advanced image segmentation techniques to identify and monitor aircraft contrails to address the challenges associated with climate change. We propose the SegX-Net architecture, a highly efficient and lightweight model that combines the DeepLabV3+, upgraded, and ResNet-101 architectures to achieve superior segmentation accuracy. We evaluated the performance of our model on a comprehensive dataset from Google research and rigorously measured its efficacy with metrics such as IoU, F1 score, Sensitivity and Dice Coefficient. Our results demonstrate that our enhancements have significantly improved the efficacy of the SegX-Net model, with an outstanding IoU score of 98.86% and an impressive F1 score of 99.47%. These results unequivocally demonstrate the potential of image segmentation methods to effectively address and mitigate the impact of air conflict on global warming. Using our proposed SegX-Net architecture, stakeholders in the aviation industry can confidently monitor and mitigate the impact of aircraft shrinkage on the environment, significantly contributing to the global fight against climate change. [ABSTRACT FROM AUTHOR]

This paper presents a novel U-Net model incorporating a hybrid attention mechanism for automating the segmentation of sub-retinal layers in Optical Coherence Tomography (OCT) images. OCT is an ophthalmology tool that provides detailed insights into retinal structures. Manual segmentation of these layers is time-consuming and subjective, calling for automated solutions. Our proposed model combines edge and spatial attention mechanisms with the U-Net architecture to improve segmentation accuracy. By leveraging attention mechanisms, the U-Net focuses selectively on image features. Extensive evaluations using datasets demonstrate that our model outperforms existing approaches, making it a valuable tool for medical professionals. The study also highlights the model's robustness through performance metrics such as an average Dice score of 94.99%, Adjusted Rand Index (ARI) of 97.00%, and Strength of Agreement (SOA) classifications like "Almost Perfect", "Excellent", and "Very Strong". This advanced predictive model shows promise in expediting processes and enhancing the precision of ocular imaging in real-world applications. [ABSTRACT FROM AUTHOR]

Background: In the medical field, medical image segmentation plays a pivotal role in facilitating disease evaluation and supporting treatment decision‐making for doctors. Recently, deep learning methods have been employed in the field of medical image segmentation. However, the manual annotation of a large number of reliable labels is a costly and time‐consuming process. Purpose: To address this challenge, a semi‐supervised learning framework is required to alleviate the burden of reliable labeling and enhance segmentation accuracy in challenging areas of medical images. Methods: Therefore, this paper presents MFA‐ICPS framework, a semi‐supervised learning framework based on the improved cross pseudo supervision (ICPS) and multi‐dimensional feature attention (MFA) modules. Medical images inevitably contain some noise that may affect the segmentation accuracy, so the proposed framework addresses this challenge by introducing noise disturbance, combining ICPS and MFA modules, and using pseudo‐segmentation maps and MFA maps to maintain the consistency at both the output and feature levels. Results: In the experiments, MFA‐ICPS framework accurately obtains the following performances on the left atrial dataset: Dice, Jaccard, 95HD, and ASD values are 90.89%$90.89\%$, 83.40%$83.40\%$, 6.00 and 1.94 mm, respectively. And on the pancreas‐CT dataset, the following performances are accurately obtained: Dice, Jaccard, 95HD, and ASD values are 79.55%$79.55\%$, 66.87%$66.87\%$, 7.67 and 1.65 mm, respectively. Conclusions: The segmentation performance of MFA‐ICPS framework on different medical datasets demonstrates its remarkable capability to significantly enhance medical image segmentation. [ABSTRACT FROM AUTHOR]

Detecting parcels accurately and efficiently has always been a challenging task when unloading from trucks onto conveyor belts because of the diverse and complex ways in which parcels are stacked. Conventional methods struggle to quickly and accurately classify the various shapes and surface patterns of unordered parcels. In this paper, we propose a parcel-picking surface detection method based on deep learning and image processing for the efficient unloading of diverse and unordered parcels. Our goal is to develop a systematic image processing algorithm that emphasises the boundaries of parcels regardless of their shape, pattern, or layout. The core of the algorithm is the utilisation of RGB-D technology for detecting the primary boundary lines regardless of obstacles such as adhesive labels, tapes, or parcel surface patterns. For cases where detecting the boundary lines is difficult owing to narrow gaps between parcels, we propose using deep learning-based boundary line detection through the You Only Look at Coefficients (YOLACT) model. Using image segmentation techniques, the algorithm efficiently predicts boundary lines, enabling the accurate detection of irregularly sized parcels with complex surface patterns. Furthermore, even for rotated parcels, we can extract their edges through complex mathematical operations using the depth values of the specified position, enabling the detection of the wider surfaces of the rotated parcels. Finally, we validate the accuracy and real-time performance of our proposed method through various case studies, achieving mAP (50) values of 93.8% and 90.8% for randomly sized and rotationally covered boxes with diverse colours and patterns, respectively. [ABSTRACT FROM AUTHOR]

This paper proposes a quasi-dense feature matching algorithm that combines image semantic segmentation and local feature enhancement networks to address the problem of the poor matching of image features because of complex distortions, considerable occlusions, and a lack of texture on large oblique stereo images. First, a small amount of typical complex scene data are used to train the VGG16-UNet, followed by completing the semantic segmentation of multiplanar scenes across large oblique images. Subsequently, the prediction results of the segmentation are subjected to local adaptive optimization to obtain high-precision semantic segmentation results for each planar scene. Afterward, the LoFTR (Local Feature Matching with Transformers) strategy is used for scene matching, enabling enhanced matching for regions with poor local texture in the corresponding planes. The proposed method was tested on low-altitude large baseline stereo images of complex scenes and compared with five classical matching methods. Results reveal that the proposed method exhibits considerable advantages in terms of the number of correct matches, correct rate of matches, matching accuracy, and spatial distribution of corresponding points. Moreover, it is well-suitable for quasi-dense matching tasks of large baseline stereo images in complex scenes with considerable viewpoint variations. [ABSTRACT FROM AUTHOR]

In recent years, UNet and its derivative networks have gained widespread recognition as major methods of medical image segmentation. However, networks like UNet often struggle with Point-of-Care (POC) healthcare applications due to their high number of parameters and computational complexity. To tackle these challenges, this paper introduces an efficient network designed for medical image segmentation called MCU-Net, which leverages ConvNeXt to enhance UNet. 1) Based on ConvNeXt, MCU-Net proposes the MCU Block, which employs techniques such as large kernel convolution, depth-wise separable convolution, and an inverted bottleneck design. To ensure stable segmentation performance, it also integrates global response normalization (GRN) layers and Gaussian Error Linear Unit (GELU) activation functions. 2) Additionally, MCU-Net introduces an enhanced Multi-Scale Convolution Attention (MSCA) module after the original UNet's skip connections, emphasizing medical image features and capturing semantic insights across multiple scales. 3)The downsampling process replaces pooling layers with convolutions, and both upsampling and downsampling stages incorporate batch normalization (BN) layers to enhance model stability during training. The experimental results demonstrate that MCU-Net, with a parameter count of 2.19 million and computational complexity of 19.73 FLOPs, outperforms other segmentation models. The overall performance of MCU-Net in medical image segmentation surpasses that of other models, achieving a Dice score of 91.8% and mIoU of 84.7% on the GlaS dataset. When compared to UNet on the BUSI dataset, MCU-Net shows an improvement of 2% in Dice and 2.9% in mIoU. [ABSTRACT FROM AUTHOR]

Accurate and reliable estimation of the pelvic tilt is one of the essential pre-planning factors for total hip arthroplasty to prevent common post-operative complications such as implant impingement and dislocation. Inspired by the latest advances in deep learning-based systems, our focus in this paper has been to present an innovative and accurate method for estimating the functional pelvic tilt (PT) from a standing anterior–posterior (AP) radiography image. We introduce an encoder–decoder-style network based on a concurrent learning approach called VGG-UNET (VGG embedded in U-NET), where a deep fully convolutional network known as VGG is embedded at the encoder part of an image segmentation network, i.e., U-NET. In the bottleneck of the VGG-UNET, in addition to the decoder path, we use another path utilizing light-weight convolutional and fully connected layers to combine all extracted feature maps from the final convolution layer of VGG and thus regress PT. In the test phase, we exclude the decoder path and consider only a single target task i.e., PT estimation. The absolute errors obtained using VGG-UNET, VGG, and Mask R-CNN are 3.04 ± 2.49, 3.92 ± 2.92, and 4.97 ± 3.87, respectively. It is observed that the VGG-UNET leads to a more accurate prediction with a lower standard deviation (STD). Our experimental results demonstrate that the proposed multi-task network leads to a significantly improved performance compared to the best-reported results based on cascaded networks. [ABSTRACT FROM AUTHOR]

Purpose: Convolution operator-based neural networks have shown great success in medical image segmentation over the past decade. The U-shaped network with a codec structure is one of the most widely used models. Transformer, a technology used in natural language processing, can capture long-distance dependencies and has been applied in Vision Transformer to achieve state-of-the-art performance on image classification tasks. Recently, researchers have extended transformer to medical image segmentation tasks, resulting in good models. Methods: This review comprises publications selected through a Web of Science search. We focused on papers published since 2018 that applied the transformer architecture to medical image segmentation. We conducted a systematic analysis of these studies and summarized the results. Results: To better comprehend the benefits of convolutional neural networks and transformers, the construction of the codec and transformer modules is first explained. Second, the medical image segmentation model based on transformer is summarized. The typically used assessment markers for medical image segmentation tasks are then listed. Finally, a large number of medical segmentation datasets are described. Conclusion: Even if there is a pure transformer model without any convolution operator, the sample size of medical picture segmentation still restricts the growth of the transformer, even though it can be relieved by a pretraining model. More often than not, researchers are still designing models using transformer and convolution operators. [ABSTRACT FROM AUTHOR]

In recent years, deep learning-based networks have been able to achieve state-of-the-art performance in medical image segmentation. U-Net, one of the currently available networks, has proven to be effective when applied to the segmentation of medical images. A Convolutional Neural Network's (CNN) performance is heavily dependent on the network's architecture and associated parameters. There are many layers and parameters that need to be set up in order to manually create a CNN, making it a complex procedure. Designing a network is made more difficult by using a variety of connections to increase the network's complexity. Evolutionary computation can be used to set the parameters of CNN and/or organize the CNN layers as an optimization strategy. This paper proposes an automatic evolutionary method for detecting an optimal network topology and its parameters for the segmentation of clinical image using Grey Wolf Optimization algorithm. Also, Bi-Directional LSTM integrated in the skip connection to extract dense feature characteristics of image by combining feature maps extracted from encoded and previous decoded path in nonlinear way (MIS-GW-U-Net-BiDCLSTM) is proposed. The experimental results demonstrate that the proposed method attains 98.49% accuracy with minimal parameters, which is much better than that of the other methods. [ABSTRACT FROM AUTHOR]

In laser powder bed fusion processes, keyholes are the gaseous cavities formed where laser interacts with metal, and their morphologies play an important role in defect formation and the final product quality. The in-situ X-ray imaging technique can monitor the keyhole dynamics from the side and capture keyhole shapes in the X-ray image stream. Keyhole shapes in X-ray images are then often labeled by humans for analysis, which increasingly involves attempting to correlate keyhole shapes with defects using machine learning. However, such labeling is tedious, time-consuming, error-prone, and cannot be scaled to large data sets. To use keyhole shapes more readily as the input to machine learning methods, an automatic tool to identify keyhole regions is desirable. In this paper, a deep-learning-based computer vision tool that can automatically segment keyhole shapes out of X-ray images is presented. The pipeline contains a filtering method and an implementation of the BASNet deep learning model to semantically segment the keyhole morphologies out of X-ray images. The presented tool shows promising average accuracy of 91.24% for keyhole area, and 92.81% for boundary shape, for a range of test dataset conditions in Al6061 (and one AliSi10Mg) alloys, with 300 training images/labels and 100 testing images for each trial. Prospective users may apply the presently trained tool or a retrained version following the approach used here to automatically label keyhole shapes in large image sets. [ABSTRACT FROM AUTHOR]

The classification of certain agricultural species poses a formidable challenge due to their inherent resemblance and the absence of dependable visual discriminators. The accurate identification of these plants holds substantial importance in industries such as cosmetics, pharmaceuticals, and herbal medicine, where the optimization of essential compound yields and product quality is paramount. In response to this challenge, we have devised an automated classification system based on deep learning principles, designed to achieve precision and efficiency in species classification. Our approach leverages a diverse dataset encompassing various cultivars and employs the Parallel Artificial Multiple Intelligence System–Ensemble Deep Learning model (P-AMIS-E). This model integrates ensemble image segmentation techniques, including U-Net and Mask-R-CNN, alongside image augmentation and convolutional neural network (CNN) architectures such as SqueezeNet, ShuffleNetv2 1.0x, MobileNetV3, and InceptionV1. The culmination of these elements results in the P-AMIS-E model, enhanced by an Artificial Multiple Intelligence System (AMIS) for decision fusion, ultimately achieving an impressive accuracy rate of 98.41%. This accuracy notably surpasses the performance of existing methods, such as ResNet-101 and Xception, which attain 93.74% accuracy on the testing dataset. Moreover, when applied to an unseen dataset, the P-AMIS-E model demonstrates a substantial advantage, yielding accuracy rates ranging from 4.45% to 31.16% higher than those of the compared methods. It is worth highlighting that our heterogeneous ensemble approach consistently outperforms both single large models and homogeneous ensemble methods, achieving an average improvement of 13.45%. This paper provides a case study focused on the Centella Asiatica Urban (CAU) cultivar to exemplify the practical application of our approach. By integrating image segmentation, augmentation, and decision fusion, we have significantly enhanced accuracy and efficiency. This research holds theoretical implications for the advancement of deep learning techniques in image classification tasks while also offering practical benefits for industries reliant on precise species identification. [ABSTRACT FROM AUTHOR]

Retinal vessel segmentation plays a vital role in the medical field, facilitating the identification of numerous chronic conditions based on retinal vessel images. These conditions include diabetic retinopathy, hypertensive retinopathy, glaucoma, and others. Although the U-Net model has shown promising results in retinal vessel segmentation, it tends to struggle with fine branching and dense vessel segmentation. To further enhance the precision of retinal vessel segmentation, we propose a novel approach called transformer dilated convolution attention U-Net (TDCAU-Net), which builds upon the U-Net architecture with improved Transformer-based dilated convolution attention mechanisms. The proposed model retains the three-layer architecture of the U-Net network. The Transformer component enables the learning of contextual information for each pixel in the image, while the dilated convolution attention prevents information loss. The algorithm efficiently addresses several challenges to optimize blood vessel detection. The process starts with five-step preprocessing of the images, followed by chunking them into segments. Subsequently, the retinal images are fed into the modified U-Net network introduced in this paper for segmentation. The study employs eye fundus images from the DRIVE and CHASEDB1 databases for both training and testing purposes. Evaluation metrics are utilized to compare the algorithm's results with state-of-the-art methods. The experimental analysis on both databases demonstrates that the algorithm achieves high values of sensitivity, specificity, accuracy, and AUC. Specifically, for the first database, the achieved values are 0.8187, 0.9756, 0.9556, and 0.9795, respectively. For the second database, the corresponding values are 0.8243, 0.9836, 0.9738, and 0.9878, respectively. These results demonstrate that the proposed approach outperforms state-of-the-art methods, achieving higher performance on both datasets. The TDCAU-Net model presented in this study exhibits substantial capabilities in accurately segmenting fine branching and dense vessels. The segmentation performance of the network surpasses that of the U-Net algorithm and several mainstream methods. [ABSTRACT FROM AUTHOR]

Semantic segmentation of Signet Ring Cells (SRC) plays a pivotal role in the diagnosis of SRC carcinoma based on pathological images. Deep learning-based methods have demonstrated significant promise in computer-aided diagnosis over the past decade. However, many existing approaches rely heavily on stacking layers, leading to repetitive computational tasks and unnecessarily large neural networks. Moreover, the lack of available ground truth data for SRCs hampers the advancement of segmentation techniques for these cells. In response, this paper introduces an efficient and accurate deep learning framework (RGGC-UNet), which is a UNet framework including our proposed residual ghost block with ghost coordinate attention, featuring an encoder-decoder structure tailored for the semantic segmentation of SRCs. We designed a novel encoder using the residual ghost block with proposed ghost coordinate attention. Benefiting from the utilization of ghost block and ghost coordinate attention in the encoder, the computational overhead of our model is effectively minimized. For practical application in pathological diagnosis, we have enriched the DigestPath 2019 dataset with fully annotated mask labels of SRCs. Experimental outcomes underscore that our proposed model significantly surpasses other leading-edge models in segmentation accuracy while ensuring computational efficiency. [ABSTRACT FROM AUTHOR]

With rapid developments in convolutional neural networks for image processing, deep learning methods based on pixel classification have been extensively applied in medical image segmentation. One popular strategy for such tasks is the encoder‐decoder‐based U‐Net architecture and its variants. Most segmentation methods based on fully convolutional networks will cause the loss of spatial and contextual information due to continuous pooling operations or strided convolution when decreasing image resolution, and make less use of contextual information and global information under different receptive fields. To overcome this shortcoming, this paper proposes a novel structure called RAAU‐Net. In our proposed RAAU‐Net structure, which is a modified U‐shaped architecture, we aim to capture high‐level information while preserving spatial information and focusing on the regions of interest. RAAU‐Net comprises three main components: a feature encoder module that utilizes a pre‐trained ResNet‐18 model as a fixed feature extractor, a multi‐receptive field extraction module that we developed, and a feature decoder module. We have tested our method on several 2D medical image segmentation tasks such as retinal nerve, breast tumor, skin lesion, lung, gland, and polyp segmentation. All the indexes of the model reached the best in the dataset of skin lesions, in which Accuracy, Precision, IoU, Recall, and Dice Score were 3.26%, 5.42%, 9.92%, 6.52%, and 5.95% higher than UNet. [ABSTRACT FROM AUTHOR]

Childhood demyelinative diseases classification (DDC) with brain magnetic resonance imaging (MRI) is crucial to clinical diagnosis. But few attentions have been paid to DDC in the past. How to accurately differentiate pediatric-onset neuromyelitis optica spectrum disorder (NMOSD) from acute disseminated encephalomyelitis (ADEM) based on MRI is challenging in DDC. In this paper, a novel architecture M-DDC based on joint U-Net segmentation network and deep convolutional network is developed. The U-Net segmentation can provide pixel-level structure information, that helps the lesion areas location and size estimation. The classification branch in DDC can detect the regions of interest inside MRIs, including the white matter regions where lesions appear. The performance of the proposed method is evaluated on MRIs of 201 subjects recorded from the Children's Hospital of Zhejiang University School of Medicine. The comparisons show that the proposed DDC achieves the highest accuracy of 99.19% and dice of 71.1% for ADEM and NMOSD classification and segmentation, respectively. [ABSTRACT FROM AUTHOR]

Early and timely fire detection within enclosed spaces notably diminishes the response time for emergency aid. Previous methods have mostly focused on singularly detecting either fire or combustible materials, rarely integrating both aspects, leading to a lack of a comprehensive understanding of indoor fire scenarios. Moreover, traditional fire load assessment methods such as empirical formula-based assessment are time-consuming and face challenges in diverse scenarios. In this paper, we collected a novel dataset of fire and materials, the Material-Auxiliary Fire Dataset (MAFD), and combined this dataset with deep learning to achieve both fire and material recognition and segmentation in the indoor scene. A sophisticated deep learning model, Dual Attention Network (DANet), was specifically designed for image semantic segmentation to recognize fire and combustible material. The experimental analysis of our MAFD database demonstrated that our approach achieved an accuracy of 84.26% and outperformed the prevalent methods (e.g., PSPNet, CCNet, FCN, ISANet, OCRNet), making a significant contribution to fire safety technology and enhancing the capacity to identify potential hazards indoors. [ABSTRACT FROM AUTHOR]

Deep learning has recently demonstrated considerable promise for a variety of computer vision tasks. However, in many practical applications, large-scale labeled datasets are not available, which limits the deployment of deep learning. To address this problem, semi-supervised learning has attracted a lot of attention in the computer vision community, especially in the field of medical image analysis. This paper analyzes existing deep semi-supervised medical image segmentation studies and categories them into five main categories (i.e., pseudo-labeling, consistency regularization, GAN-based methods, contrastive learning-based methods, and hybrid methods). Afterward, we empirically analyze several representative methods by conducting experiments on two common datasets. Besides, we also point out several promising directions for future research. In summary, this paper provides a comprehensive introduction to deep semi-supervised medical image segmentation, aiming to provide a reference and comparison of methods for researchers in this field. [ABSTRACT FROM AUTHOR]