Search

Seismic fault detection is indispensable for exploring reservoirs of hydrocarbons, and a considerable amount of research has thus been devoted to it. With the rapid development of deep learning in recent years, researchers have begun using convolutional neural networks (CNNs) to identify seismic faults. However, it remains challenging for geophysical interpreters to quickly train an efficient model on a limited number of samples for fault interpretation. In this letter, we propose a workflow that uses a CNN-based method of semantic segmentation to interpret faults by using a small training set. It requires only the selection of some 2-D seismic sections from seismic volume data for interpretation and labeling to be trained to predict faults in the entire area considered. We simplified the VGG16 model to reduce training time and modified it to improve its performance. We used convolution layers instead of fully connected layers at the end of the network to implement the end-to-end classification of seismic images, and adopted dilation convolution to increase the receptive field and hybrid dilation convolution to avoid problems in it. We also applied the atrous spatial pyramid pooling (ASPP) module to further enhance the effects of segmentation. The results were subjected to postprocessing for refinement. The promising performance of the proposed method was verified on a set of real seismic data.

Deep learning for 3-D point cloud perception has been a very active research topic in recent years. A current trend is toward the combination of the semantically strong and the fine-grained information from different scales of intermediate representations to boost network generalization power and robustness against scale variation. One prominent challenge is how to effectively conduct the allocation of multiple scales of information. In this letter, we propose a module, named adaptive pyramid context fusion (APCF), to adaptively capture scales of contextual information from a multiscale feature pyramid for the point cloud. The APCF module reweights and aggregates the features from different levels in the feature pyramid via a softmax attention strategy. The allocation of information is adaptively conducted level by level from bottom to up first and then from top to bottom. To ensure both effectiveness and efficiency, we propose a multiscale context-aware network APCF-Net through applying our proposed APCF to the PointConv architecture. Experiments demonstrate that APCF-Net surpasses its vanilla counterpart by a large margin both in effectiveness and efficiency. Especially, APCF-Net outperforms state-of-the-art approaches on 3-D object classification and semantic segmentation task, with the overall accuracy of 93.3% on ModelNet40 and mIoU of 63.1% on ScanNet V2 online test.

Modern autonomous vehicles are required to perform various visual perception tasks for scene construction and motion decision. The multiobject tracking and instance segmentation (MOTS) are the main tasks since they directly influence the steering and braking of the car. Implementing both tasks using a multitask learning neural network presents significant challenges in performance and complexity. Current work on MOTS devotes to improve the precision of the network with a two-stage tracking by detection model, which is difficult to satisfy the real-time requirement of autonomous vehicles. In this article, a real-time multitask network named YolTrack based on one-stage instance segmentation model is proposed to perform the MOTS task, achieving an inference speed of 29.5 frames per second (fps) with slight accuracy and precision drop. The YolTrack uses ShuffleNet V2 with feature pyramid network (FPN) as a backbone, from which two decoders are extended to generate instance segments and embedding vectors. Segmentation masks are used to improve the tracking performance by performing logic AND operation with feature maps, proving that foreground segmentation plays an important role in object tracking. The different scales of multiple tasks are balanced by the optimized geometric mean loss during the training phase. Experimental results on the KITTI MOTS data set show that YolTrack outperforms other state-of-the-art MOTS architectures in real-time aspect and is appropriate for deployment in autonomous vehicles.

Semantic segmentation is a valuable yet challenging research in the Internet of Things (IoT), especially for some real-time and resource-constrained applications. Recently we witness a strong tendency of fusing multi-level features or multi-scale context information to achieve promising segmentation performance . However, we find that existing literature has at least one of the following issues: 1) relying on more resource-consuming feature extraction operations, e.g., standard convolution with large kernels, for multiple information fusion; 2) seldom considering that how to narrow the semantic gap between multi-layer features, leading to sub-optimal performance. To tackle these issues, we propose a novel IoT-perceptive Dual Feature Fusion Network (DFFNet) for semantic segmentation, which aims to leverage multi-level features and multi-scale context information in an efficient yet effective manner. Specifically, the multi-level feature fusion module (MFFM), which enhances the semantic consistency between multi-level features by two attention refinement blocks, is proposed to exploit multi-layer features for jointly learning spatial and semantic information with small overheads. Moreover, a multi-scale component termed as lightweight semantic pyramid module (LSPM) is presented to improve the efficiency of context encoding by depthwise factorized convolution operations . Extensive experimental results on benchmarks have demonstrated that DFFNet achieves better performance than existing advanced methods.

The retinal vessel is the only microvascular network that can be directly and non‐invasively observed in humans. Cardiovascular and cerebrovascular diseases, such as diabetes, hypertension, can lead to structural changes of the retinal microvascular network. Therefore, it is of great significance to study effective retinal vessel segmentation methods and assist doctors in early diagnoses with quantitative results for vascular networks. In this study, we propose a novel convolutional neural network named feature pyramid U‐Net (FPU‐Net) that extracts multiscale representations by constructing two feature pyramids both on the encoder and the decoder of U‐Net. In this representation, objects features with different size like micro‐vessels and pathology will be fused for better vessel segmentation. The experimental results show that compared with state‐of‐the‐art methods, FPU‐Net is superior in terms of accuracy, sensitivity, F1‐score, and area under the curve and capable of stronger domain generalisation across different datasets.

Real-time semantic segmentation aims to generate high-quality prediction in limited time. Recently, with the development of many related potential applications, such as autonomous driving, robot se...

Convolutional neural networks (CNNs) have facilitated impressive improvements in the semantic segmentation of very high-resolution (VHR) remote sensing images. The success of semantic segmentation depends on an effective receptive field (RF) large enough to cover the entire object. Popular methods to enlarge the effective RF include dilated filters, subsampling operations, and stacking layers. Unfortunately, the methods are inefficient or able to cause grid artifacts. Moreover, although the object sizes vary greatly in remote sensing images, the size of the RF cannot reach a compromise between small and large objects. To tackle these problems, we propose adaptive effective receptive convolution (AERFC) for VHR remote sensing images. AERFC adaptively controls the sampling location of convolution and automatically adjusts the effective RF without significantly increasing the parameter number and computational cost. Thus, AERFC reduces the training difficulty, decreases overfitting risk, and reserves details in VHR images. AERFC is also integrated with spatial pyramid pooling (SPP) to aggregate diverse multiscale features for exploring contextual information. Experimental results of the quantitative and qualitative evaluation over four benchmark data sets show that AERFC outperforms state-of-the-art methods.

A prerequisite for solving many agricultural problems is to accurately estimate the area affected by crop disasters and its severity rating. In this letter, we propose a pipeline to segment the drought area and distinguish the severity rating of the maize on RGB images accessed by an unmanned aerial vehicle (UAV) through a semantic segmentation method based on deep learning. First, the ground truth is created through expert evaluation and visual interpretation with the aid of the Normalized Difference Vegetation Index (NDVI). The neural network structure that was used is based on U-Net. Some structural and parameter improvements on U-net were made using SE-ResNeXt-50 as the backbone with the atrous spatial pyramid pooling (ASPP) module. By using RGB images as the input of the neural network for training, the final trained network can work on RGB images captured by a consumer UAV. The experimental results showed that our pipeline achieved an F1-score of 0.9034 and a Jaccard index of 0.8287 on the test set.

Proposal-free instance segmentation methods mainly generate instance-agnostic semantic segmentation labels and instance-aware features to group pixels into different object instances. However, previous methods mostly employ separate modules for these two sub-tasks and require multiple passes for inference. In addition to the lack of efficiency, previous methods also failed to perform as well as proposal-based approaches. To this end, this work proposes a single-shot proposal-free instance segmentation method that requires only one single pass for prediction. Our method is based on learning an affinity pyramid, which computes the probability that two pixels belong to the same instance in a hierarchical manner. Moreover, incorporating with the learned affinity pyramid, a novel cascaded graph partition (CGP) module is presented to fuse the two predictions and segment instances efficiently. As an additional contribution, we conduct an experiment to demonstrate the benefits of proposal-free methods in capturing detailed structures from finely annotated training examples. Our approach is evaluated on the Cityscapes and COCO datasets and achieves state-of-the-art performance.

In this paper, we explore an issue that is to detect vehicle headlights from the nighttime traffic surveillance images with highly reflections. In the night, reflections on the road (water) surface, vehicles bodies, or some reflective objects (such as lane markings, traffic signs) will interfere the headlight detection seriously. Although the existing methods have achieved good results, however, most of them failed to detect the headlight when headlights are far from camera. In order to solve the issue, we propose a novel method for vehicle headlights detection. The proposed method makes full use of the brightness and gradient information of the headlights in the night. First, we propose an effective region-of-interest (ROI) segmentation method which is based on multi-scale local saliency detection. The method pre-serve faint or small-sized objects and retain the original shape of the object to the greatest extent. Then, we compute the pyramid histogram of oriented gradients (PHOG) features, which are used to train support vector machine (SVM) classifier. Finally, the extracted bright blocks are classified according to the pre-trained SVM classifier. Experimental results and quantitative evaluations in different scenes demonstrate that our proposed method can achieve a better result compared with previous methods.

Recently, object detection has been a vital aspect in the vision community, while scale variation of objects in images or videos usually brings challenge for performance improvement. To combat this problem, conventional paradigms generally adopt image pyramid or Feature Pyramid Network (FPN) to process objects at different scales. However, existing multi-scale deep convolution neural networks mostly set different scales in a heuristic way, which may introduce inconsistency between the region of interest and the semantic scope. In this paper, we propose an innovative paradigm called Consistent Scale Normalization (CSN) to weaken the influence of scale variation for object detection. The proposed CSN can realize a consistent compression for the scale space of objects, in both training and testing phases. Extensive experimental testing is performed on COCO object detection benchmark in comparison with several state-of-the-art methods. In addition to object detection, experiments on instance segmentation and multi-task human pose estimation are also conducted. Furthermore, the CSN paradigm is beneficial to reduce the difficulty of network learning. The results verify the effectiveness and superiority of the CSN paradigm.

Semantic segmentation is a task that covers most of the perception needs of intelligent vehicles in an unified way. Recent studies witnessed that attention mechanisms achieve impressive performance in computer vision task. Current attention mechanisms based segmentation methods differ with each other in position and form of the attention mechanism, and perform differently in practice. This paper firstly introduces the effectiveness of multi-scale context features and attention mechanisms in segmentation tasks. We find that multi-scale and channel attention can play a vital role in constructing effective context features. Based on this analysis, this paper proposes an efficient attention pyramid network (EAPNet) for semantic segmentation. Specifically, to efficient handle the problem of segmenting objects at multiple scales, we design efficient channel attention pyramid (ECAP) which employ atrous convolution with channel attention in cascade or in parallel to capture multi-scale context by using multiple atrous rates. Furthermore, we propose a residual attention fusion block (RAFB), whose purpose is to simultaneously focus on meaningful low-level feature maps and spatial location information. At the same time, we will explore different channel attention modules and spatial attention modules, and describe their impact on network performance. We empirically evaluate our EAPNet on two semantic segmentation datasets, including PASCAL VOC 2012 and Cityscapes datasets. Experimental results show that without MS COCO pre-training and any post-processing, EAPNet achieved 81.7% mIoU on the PASCAL VOC 2012 validation set. With deeplabv3+ as the benchmark, EAPNet improve the model performance of more than 1.50% mIoU.

Disease is one of the main factors affecting crop growth. How to reflect the external morphological features of the disease and completely retain the color and texture information of the disease area is one of the key research issues for crop disease segmentation. Meanwhile, aiming at the problem of low segmentation accuracy with traditional convolution neural network-based methods in the crop disease leaf image, this paper proposes a spatial pyramid-oriented encoder-decoder cascade convolution neural network for crop disease leaf segmentation. The network consists of a region disease detection network and a region disease segmentation network. Region disease detection network is a kind of network combining cascade convolution neural network with spatial pyramid. This method connects the three-level convolution neural network model, where the structure of the three-level neural network model varies from simple to complex. Different crop disease leaf features are extracted from the different neural network levels. And images are screened to complete the detection of crop disease leaf. What’s more, a space pyramid pooling layer is added to each network level. This pooling strategy does not require fixed size input, which increases the size selection of input model. The region segmentation network is established based on the Encoder-Decoder structure. The multi-scale convolution kernel is used to improve the local receptive field of the original convolution kernel and accurately segment the crop disease leaf area. Finally, we conduct experiments on the crop disease leaf images under different conditions, the results show that the proposed method has higher segmentation accuracy. In terms of Precision, Correct segmentation, over-segmentation and under-segmentation indexes, etc., the average values of proposed method are more than 90%. The average dice similarity coefficient is over 95% under different background. Moreover, it can meticulously reflect the external morphological features of the crop disease leaf and relatively better retain the color and texture information.

Automatic surface defect inspection (ASDI) is a crucial and challenging problem in industry because it affects the quality and efficiency of production greatly. Deep learning-based methods achieve promising improvements for surface detection, but they rely on a massive training data set that is impractical in industry. In this study, we propose a generic method that works effectively even on the small size training data set. Especially, we introduce a ResMask generative adversarial network (GAN) framework that is a residual GAN to expand the insufficient defect data sets. Meanwhile, the existing inspection data sets are so much easier to detect than the data sets in the real industrial scenarios that new industrial surface defect data sets containing more diverse and challenging images are established. Then, a coarse-to-fine module (CFM) that consists of a coarse detection subnetwork and a fine segmentation subnetwork is proposed for the needs of fast detection for high-resolution images. In the coarse detection stage, spatial pyramid pooling (SPP) is utilized to increase the receptive field of the network, reduce the false detection rate, and determine the approximate location of defects. In the fine segmentation stage, the receptive field of the network is enlarged by atrous SPP (ASPP), and skip links that incorporate low-level with high-level features achieve pixelwise precision on defect segmentation. Finally, our algorithm has achieved state-of-the-art results in DAGM, HR, and WB data sets (0.859, 0.761, and 0.805, respectively) according to MIoU. It achieves an average processing time of 44.4 ms on the test images with a resolution of $512 \,\,\times \,\, 512$ and 131.6 ms for $2048 \times 2048$ images. On the DAGM data set, the detection accuracy of mean intersection over union (MIoU) reaches 0.859.

Medical segmentation is an important but challenging task with applications in standardized report generation, remote medicine and reducing medical exam costs by assisting experts. In this paper, we exploit time sequence information using a novel spatio-temporal recurrent deep learning network to automatically segment the thyroid gland in ultrasound cineclips. We train a DeepLabv3+ based convolutional LSTM model in four stages to perform semantic segmentation by exploiting spatial context from ultrasound cineclips. The backbone DeepLabv3+ model is replicated six times and the output layers are replaced with convolutional LSTM layers in an atrous spatial pyramid pooling configuration. Our proposed model achieves mean intersection over union scores of 0.427 for cysts, 0.533 for nodules and 0.739 for thyroid. We demonstrate the potential application of convolutional LSTM models for thyroid ultrasound segmentation.

The convolutional neural network (CNN)-based multi-focus image fusion methods which learn the focus map from the source images have greatly enhanced fusion performance compared with the traditional methods. However, these methods have not yet reached a satisfactory fusion result, since the convolution operation pays too much attention on the local region and generating the focus map as a local classification (classify each pixel into focus or de-focus classes) problem. In this article, a global-feature encoding U-Net (GEU-Net) is proposed for multi-focus image fusion. In the proposed GEU-Net, the U-Net network is employed for treating the generation of focus map as a global two-class segmentation task, which segments the focused and defocused regions from a global view. For improving the global feature encoding capabilities of U-Net, the global feature pyramid extraction module (GFPE) and global attention connection upsample module (GACU) are introduced to effectively extract and utilize the global semantic and edge information. The perceptual loss is added to the loss function, and a large-scale dataset is constructed for boosting the performance of GEU-Net. Experimental results show that the proposed GEU-Net can achieve superior fusion performance than some state-of-the-art methods in both human visual quality, objective assessment and network complexity.

Accurate and efficient extraction of road marking plays an important role in road transportation engineering, automotive vision, and automatic driving. In this article, we proposed a dense feature pyramid network (DFPN)-based deep learning model, by considering the particularity and complexity of road marking. The DFPN concatenated its shallow feature channels with deep feature channels so that the shallow feature maps with high resolution and abundant image details can utilize the deep features. Thus, the DFPN can learn hierarchical deep detailed features. The designed deep learning model was trained end to end for road marking instance extraction with mobile laser scanning (MLS) point clouds. Then, we introduced the focal loss function into the optimization of deep learning model in road marking segmentation part, to pay more attention to the hard-classified samples with a large extent of background. In the experiments, our method can achieve better results than state-of-the-art methods on instance segmentation of road markings, which illustrated the advantage of the proposed method.

Small fasteners are widely used in a freight train. However, due to the complex working environment, the missing of fasteners always happens, causing traffic accidents and property loss. Thus, detection of the missing of small fasteners is essential for the safety of freight trains. This article takes three critical missing small fasteners as examples, pin-missing, bolt-missing, and rivet-missing defects, and proposes a two-stage small defect detection model. First, a copy–pasting method is used to augment the defects in the training image, then the ResNet-101 backbone is adopted to extract defect features, and a feature pyramid network is used to construct a feature pyramid. After obtaining defect features, an improved region proposal network is introduced to generate defect proposals. Finally, a fully convolutional neural network is trained to operate pixelwise segmentation. The experimental results on manually collected fastener defect data set show that our model possesses high precision and efficiency on the inspection of pin-missing, bolt-missing, and rivet-missing defects. Besides, the proposed model could be used for similar small defects detection.

Surface defect segmentation of no-service rail is important for its quality assessment. There are several challenges of uneven illumination, complex background, and difficulty of sample collection for no-service rail surface defects (NRSDs). In this article, we propose an acquisition scheme with two lamp light and color scan line charge-coupled device (CCD) to alleviate uneven illumination. Then, a multiple context information segmentation network is proposed to improve NRSD segmentation. The network makes full use of context information based on dense block, pyramid pooling module, and multi-information integration. Besides, the attention mechanism is applied to optimize extracted information by filtering noise. For the problem of real sample shortage, we propose to utilize artificial samples to train the network. And an NRSD data set NRSD-MN is built with artificial NRSDs and natural NRSDs. Experimental results show that our method is feasible and has a good segmentation effect on artificial and natural NRSDs.

The segmentation effect of welding zone is vulnerable to image quality and target shape, which brings difficulty for the radiographic inspection of water-cooled pipe. A boundary-aware semantic segmentation algorithm is proposed to improve the precision and robustness of the welding zone inspection system. First, a convolutional neural network and atrous spatial pyramid pooling are employed to generate the feature map of the input image. Second, a boundary distance field (BDF) regression module is designed to predict the distance fields of the inner and outer boundaries, which can reflect the continuous boundary information of welding zone. Then, spatial attention weight is calculated according to the regressed boundary distance fields, and the calculated weight is fused into the feature map to enhance the significance of welding zone. Finally, the segmentation mask is predicted based on the fused feature map. Experiments indicate that the guidance of boundary information in the semantic segmentation algorithm can effectively reduce the interference of nonweld zones. The segmentation accuracy of the welding zone is 93.70%, and average detection time of a single image is less than 11 ms.

Accurate and reliable segmentation of the prostate, its inner and surrounding structures in magnetic resonance (MR) images, is of essential importance for image-guided prostate interventions and treatments. Further clinical demand is the automated segmentation of central gland (CG) and peripheral zone (PZ). Although T2-weighted (T2W) images can show prostate tissues more clearly, the lack of contrast between PZ and other surrounding tissues in T2W images increases the difficulty of PZ segmentation. In this context, it is necessary to consider using multi-parametric MR images to obtain complementary information for prostate zonal segmentation. In this paper, we propose a 3D multi-scale discriminative network with pyramid attention module (PAM) and residual refinement block (RRB) for automated and accurate segmentation of CG and PZ using bi-parametric (T2W and apparent diffusion coefficient) MR images. One of the major difficulties in prostate MR image segmentation is the ambiguous edge between the prostate and other surrounding anatomical structures, which is reflected in some specific directions. Therefore, we design a multi-directional edge loss to help the network focus on the multi-directional edge information of foreground areas. For the Prostate Multi-parametric MRI (PROMM) dataset, our proposed model achieved Dice similarity coefficient of 0.908 at CG and 0.785 at PZ. The average boundary distance obtained by our model is 1.397 mm at CG and 3.891 mm at PZ. For the NCI-ISBI dataset, our method greatly improves the Dice similarity coefficient at PZ, reaching 0.806 and achieved the Dice similarity coefficient of 0.901 at CG. The experimental results on two different MR prostate datasets demonstrate that our model is more sensitive to object boundaries and outperforms other state-of-the-art methods. The visualization of feature map activations in PAM shows that the proposed model can capture multi-scale discriminative features effectively.

Direct regression methods have demonstrated their success on various multi-oriented benchmarks for scene text detection due to the high recall rate for small targets and the direct regression for text boxes. However, too many false positive candidates and inaccurate position regression still limit the performance of these methods. In this paper, we propose an end-to-end method by introducing position-sensitive segmentation into the direct regression method to overcome these shortcomings. We generate the ground truth of position-sensitive segmentation maps based on the information of text boxes so that the position-sensitive segmentation module can be trained synchronously with the direct regression module. Besides, more information about the relative position of text is provided for the network through the training of position-sensitive segmentation maps, which improves the expressiveness of the network. We also introduce spatial pyramid of position-sensitive segmentation into the proposed method considering the huge differences in sizes and aspect ratios of scene texts and we propose position-sensitive COI(Corner area of Interest) pooling into the proposed method to speed up the inference. Experiments on datasets ICDAR2015, MLT-17 and COCO-Text demonstrate that the proposed method has a comparable performance with state-of-the-art methods while it is more efficient. We also provide abundant ablation experiments to demonstrate the effectiveness of these improvements in our proposed method.

Blur detection aims to detect the regions where the image is blur and identifies the blurred regions accurately. Since the traditional hand-crafted feature based methods usually are not robust enough to handle various complex scenarios, the study of blur detection problem remains to be a challenging task in terms of the accuracy and effectiveness of blur separation. In this paper, instead of seeking a new hand-crafted feature often seen in common approaches, we present a new deep feature-learning method by using feature pyramid network via recurrent Distinction Enhanced Block modules to solve the blur detection problem. We show that the newly introduced Distinction Enhanced Block is able to merge the high-level semantic information with the low-level details effectively, while at the same time, it is capable to keep a desirable identification between the blurry and clear regions. We also develop a new boundary penalty term to refine the boundaries of our results, which leads to more accurate detection and segmentation, compared with current available results in literature. Furthermore, due to the lacking of public datasets for blur detection problems, we have established a new blur detection dataset which may enrich the experimental benchmarks for testing. In terms of efficiency associated with the competitive performance, in particular, our model is shown to have best-in-class evaluation speed, about 0.14 second to evaluate an input image in regular personal laptop. The performance evaluations on both CUHK dataset and our SZU-BD dataset validate that the proposed model outperforms various existing state-of-the-arts methods through commonly used metric indexes.

PURPOSE Ultrasound image segmentation is a challenging task due to a low signal-to-noise ratio and poor image quality. Although several approaches based on the convolutional neural network (CNN) have been applied to ultrasound image segmentation, they have weak generalization ability. We propose an end-to-end, multiple-channel and atrous CNN designed to extract a greater amount of semantic information for segmentation of ultrasound images. METHOD A multiple-channel and atrous convolution network is developed, referred to as MA-Net. Similar to U-Net, MA-Net is based on an encoder-decoder architecture and includes five modules: the encoder, atrous convolution, pyramid pooling, decoder, and residual skip pathway modules. In the encoder module, we aim to capture more information with multiple-channel convolution and use large kernel convolution instead of small filters in each convolution operation. In the last layer, atrous convolution and pyramid pooling are used to extract multi-scale features. The architecture of the decoder is similar to that of the encoder module, except that up-sampling is used instead of down-sampling. Furthermore, the residual skip pathway module connects the subnetworks of the encoder and decoder to optimize learning from the deeper layer and improve the accuracy of segmentation. During the learning process, we adopt multi-task learning to enhance segmentation performance. Five types of datasets are used in our experiments. Because the original training data are limited, we apply data augmentation (e.g., horizontal and vertical flipping, random rotations, and random scaling) to our training data. We use the Dice score, precision, recall, Hausdorff distance (HD), average symmetric surface distance (ASD), and root mean square symmetric surface distance (RMSD) as the metrics for segmentation evaluation. Meanwhile, Friedman test was performed as the nonparametric statistical analysis to evaluate the algorithms. RESULTS For the datasets of brachia plexus (BP), fetal head, and lymph node segmentations, MA-Net achieved average Dice scores of 0.776, 0.973, and 0.858, respectively; with average precisions of 0.787, 0.968, and 0.854, respectively; average recalls of 0.788, 0.978, and 0.885, respectively; average HDs (mm) of 13.591, 10.924, and 19.245, respectively; average ASDs (mm) of 4.822, 4.152, and 4.312, respectively; and average RMSDs (mm) of 4.979, 4.161, and 4.930, respectively. Compared with U-Net, U-Net++, M-Net, and Dilated U-Net, the average performance of the MA-Net increased by approximately 5.68%, 2.85%, 6.59%, 36.03%, 23.64%, and 31.71% for Dice, precision, recall, HD, ASD, and RMSD, respectively. Moreover, we verified the generalization of MA-Net segmentation to lower grade brain glioma MRI and lung CT images. In addition, the MA-Net achieved the highest mean rank in the Friedman test. CONCLUSION The proposed MA-Net accurately segments ultrasound images with high generalization, and therefore, it offers a useful tool for diagnostic application in ultrasound images.

Accurate and automatic segmentation of medical images is a crucial step for clinical diagnosis and analysis. The convolutional neural network (CNN) approaches based on the U-shape structure have achieved remarkable performances in many different medical image segmentation tasks. However, the context information extraction capability of single stage is insufficient in this structure, due to the problems such as imbalanced class and blurred boundary. In this paper, we propose a novel Context Pyramid Fusion Network (named CPFNet) by combining two pyramidal modules to fuse global/multi-scale context information. Based on the U-shape structure, we first design multiple global pyramid guidance (GPG) modules between the encoder and the decoder, aiming at providing different levels of global context information for the decoder by reconstructing skip-connection. We further design a scale-aware pyramid fusion (SAPF) module to dynamically fuse multi-scale context information in high-level features. These two pyramidal modules can exploit and fuse rich context information progressively. Experimental results show that our proposed method is very competitive with other state-of-the-art methods on four different challenging tasks, including skin lesion segmentation, retinal linear lesion segmentation, multi-class segmentation of thoracic organs at risk and multi-class segmentation of retinal edema lesions.

Semantic scene segmentation has become a key application in computer vision and is an essential part of intelligent transportation systems for complete scene understanding of the surrounding environment. While several methods based on deep fully Convolutional Neural Network (CNN) have been emerging, there are two main challenges: (i) They mainly focus on improvement of the accuracy than efficiency. (ii) They assume structured driving environment like in USA and Europe. While most of the current works focus on the well structured driving environment, we focus our research on India Driving Dataset (IDD) which contains data from unstructured traffic scenario. In this paper, we propose modifications in the DeepLabV3+ framework by using lower atrous rates in Atrous Spatial Pyramid Pooling (ASPP) module for dense traffic prediction. We propose to use dilated Xception network as the backbone for feature extraction. A lightweight segmentation framework is also presented by exploring the effectiveness of MobileNetV2 architecture, which achieves competitively high accuracy and is much smaller than other state-of-art architectures. The performance is evaluated in terms of mean Intersection over Union (mIoU) on 26 fine grained classes of IDD. Our proposed model with 24 M parameters achieves 68.41 mIoU on test set and efficient mobile model achieves mIoU of 61.6 by reducing the parameters to 2.2 M only.

Convolution filters play important roles in improving the abstraction ability of segmentation networks due to its method of fusing features from all input channels. Such a method pays little attention to protecting the channel-wise features which is important in preserving the detail features in the final result. Though a few works has discussed this problem, the function of channel-wise features is still not sufficiently exploited. For investigating the beneficial effect of channel-wise feature on improving details representation ability, we design a Hybrid Pyramid Pooling (HPP) structure to explicitly exploit the channel-wise features. Through utilizing different scales of depthwise dilated convolution, segmentation networks can fully utilize the superiority of channel-wise features to improve the details representation ability. For deeply developing the advantages of the channel-wisely calculated features by HPP, we design another Stacked Pyramid Structure (SPS), which contains stacked pyramid pooling structures. The SPS distributes sufficient sampling points in each finely divided sampling area, and fuses features from a large range of receptive fields to produce rich feature representation. Finally, the combination of HPP and SPS can not only maintains both the advantages of features calculated from a particular channel and all input channels, but also provides sufficiently enriched receptive fields for depicting details in the final result.

Road segmentation from remote sensing images is a critical task in many applications. In recent years, various approaches, particularly deep learning-based methods, have been proposed for accurate road segmentation. However, most existing road segmentation methods always obtain unsatisfactory results (e.g., heterogeneous pixels) due to the complex backgrounds and view occlusions of buildings and trees around a road; consequently, road segmentation remains a challenging problem. In this study, we propose a novel global context based dilated convolutional neural network (GC-DCNN) to address the aforementioned problem. The structure of GC-DCNN is similar to that of UNet. In particular, building the encoder of GC-DCNN with three residual dilated blocks is suggested to further enlarge the effective receptive field and learn additional discriminative features. Thereafter, a pyramid pooling module is used to capture the multiscale global context features and fuse them to achieve stronger feature representation. The decoder network upsamples the fused features to the same size as the input image, combining the high-resolution features with the contracting path of the encoder network. Moreover, the dice coefficient loss is adopted as the loss function. This function differs from those in most previous studies but is more suitable for road segmentation. Extensive experimental results on two benchmark datasets compared with several baseline models demonstrate the superiority of the proposed GC-DCNN algorithm.

This article reports the development of a pure deep learning method for segmenting concrete cracks in images. The objectives are to achieve the real-time performance while effectively negating a wide range of various complex backgrounds and crack-like features. To achieve the goals, an original convolutional neural network is proposed. The model consists of standard convolutions, densely connected separable convolution modules, a modified atrous spatial pyramid pooling module, and a decoder module. The semantic damage detection network (SDDNet) is trained on a manually created crack dataset, and the trained network records the mean intersection-over-union of 0.846 on the test set. Each test image is analyzed, and the representative segmentation results are presented. The results show that the SDDNet segments cracks effectively unless the features are too faint. The proposed model is also compared with the most recent models, which show that it returns better evaluation metrics even though its number of parameters is 88 times less than in the compared models. In addition, the model processes in real-time (36 FPS) images at 1025 × 512 pixels, which is 46 times faster than in a recent work.

Automated nuclear segmentation in histopathological images is a prerequisite for a computer-aided diagnosis framework. However, it remains a challenging problem due to the nucleus occlusion or overlapping, shape variation, and image background complexity. Recently, deep learning techniques are widely used in analyzing digital histopathology. We present a computerized image-based method for automatically segmenting nuclei using an integration of a deep learning model and an improved concave point detection algorithm. A modified atrous spatial pyramid pooling U-Net (ASPPU-Net) is derived to capture multi-scale nuclei features and obtain nuclei context information without reducing the spatial resolution of feature map. A weighted binary cross entropy loss function with Dice loss function is used to better handle the data unbalance problem. An accelerated concave point detection method allows to effectively and accurately segmenting highly overlapping nuclei. Our ASPPU-Net based method was tested on four independent data cohorts and achieved the highest Dice similarity coefficient of 0.83, and pixel wise accuracy of 0.95. The experimental results suggested that the combination of ASPPU-Net model and concave point detection method was able to gain improved performance in separating both isolated and touching clustered nuclei in histopathology.

Recently, scene text detection has become an active research field, which is an essential component of scene text reading. Especially, segmentation-based methods are commonly used, since the segmentation results can describe text of arbitrary shape. However, curve texts have a diversity of shapes, scales and orientations, which are difficult to locate, so the detector requires to adjust the local receptive fields size adaptively, which can aggregate multi-scale spatial information to accurately locate the curve text instance. Moreover, the low-level features are critical for localizing large text instances. When using Feature Pyramid Network (FPN) for multi-scale feature fusion, it will prevent the flow of accurate localization signals due to the long path from low-level to top-level. In order to solve these two problems, this paper proposes an Adaptive Convolution and Path Enhancement Pyramid Network (ACPEPNet), which can more accurately locate the text instances with arbitrary shapes. Firstly, an Adaptive Convolution Unit is introduced to improve the ability of backbone to aggregate multi-scale spatial information at the same stage. Specially, this unit is a lightweight component and without the cost of computations, based on this component we present a backbone network for text features extraction. Secondly, the original FPN structure is redesigned to build a short path from the low-level to top-level, in this way, we modify the path from one-way flow to two-way flow and add original features to the final stage of information fusion. Experiments on CTW1500, Total-Text, ICDAR 2015 and MSRA-TD500 validate the robustness of the proposed method. When there is no bells and whistles, this method achieves an F-measure of 80.8% without external training data on CTW1500.

Traditional deep convolutional neural networks for fully automatic brain tumor segmentation have two problems: spatial information loss caused by both the repeated pooling/striding and weak ability of multi-scale lesion processing. To overcome the first problem, we use a 3D atrous-convolution with a single stride to replace pooling/striding and build the backbone for feature learning. For the second problem, a 3D atrous-convolution feature pyramid is designed and added to the end of the backbone. By integrating with contextual features, this structure improves the discriminating ability of the overall model to segment tumors with various sizes. Finally, a 3D fully connected Conditional Random Field is constructed as a post-processing step for the network’s output to obtain structural segmentation of both the appearance and spatial consistency. Abundant ablation experiments carried on Magnetic Resonance Imaging datasets demonstrate that lossless feature computation and multi-scale information fusion driven by our method are feasible to address the above problems. Compared with the state-of-the-art methods on the public benchmarks, our method achieves competitive performance and can be efficiently implemented into the clinical medical application.

Purpose Automated segmentation could improve the efficiency of modeling-based pelvic organ prolapse (POP) evaluations. However, segmentation performance is limited by the blurry soft tissue boundaries. In this study, we aimed to present a hybrid solution for uterus, rectum, bladder, and levator ani muscle segmentation by combining a convolutional neural network (CNN) and a level set method. Methods We used 24 sagittal pelvic floor magnetic resonance (MR) series from six anterior vaginal prolapse and six posterior vaginal prolapse subjects (a total 528 MR images). The stress MR images were performed both at rest and at maximal Valsalva. We assigned 264 images for training, 132 images for validation, and 132 images for testing. A CNN was designed by introducing a multi-resolution features pyramid module (MRFP) into an encoder-decoder model. Depth separable convolution and pretraining were used to improve model convergence. Multiclass cross entropy loss and multiclass Dice loss were used for model training. The dice similarity coefficient (DSC) and average surface distance (ASD) were used for evaluating the segmentation results. To prove the effectiveness of our model, we compared it with advanced segmentation methods including Deeplabv3+, U-Net, and FCN-8s. The ablation study was designed to quantify the contributions of MRFP, the encoder network, and pretraining. Besides, we investigated the working mechanism of MRFP in the segmentation network by comparing our model with three of its variants. Finally, the level set method was used to improve the CNN model further. Results Dice loss showed better segmentation performance than multiclass cross entropy loss. MRFP was efficacious for different encoder networks. With MRFP, U-Net and U-Net-X (X represents Xception encoder network) have improved the DSC, on average by 6.8 and 5.3 points. Compared with different CNN models, our model achieved the highest average DSC of 65.6 points and the lowest average ASD of 2.9 mm. With the level set method, the DSC of our model improved to 69.4 points. Conclusions MRFP proved to be effective in addressing the blurry soft tissue boundary problem on pelvic floor MR images. A hybrid solution based on CNN and level set method was presented for pelvic organ segmentation both at rest and at maximal Valsalva; with this method, we achieved state-of-the-art results.

Because of the large variation in appearance, the existence of artifacts, the low contrast, skin lesion segmentation remains a challenging task. In this paper, we propose a novel Scale Attention based Atrous Spatial Pyramid Pooling (Scale-Att-ASPP) module for skin lesion segmentation with attentive boundary aware. Our network is based on the Generative Adversarial Network (GAN), which includes the segmentation network and the critic network. In the segmentation network, we design the Scale-Att-ASPP module to automatically select the optimal scale of the skin lesion feature of the intermediate convolution layer (Inter-CL) in the encoding path, meanwhile, the irrelevant artifacts features are automatically diminished without using complex pre-processing. After introducing the output of the Scale-Att-ASPP module to the same level layer in the decoding path through skip connection in pixel-wise addition way, the more meaningful semantic segmentation is gained. The Jaccard distance loss is employed to solve the problem of label imbalance in skin lesion segmentation. Our network is adversarially trained on ISBI 2017 dataset by the multi-scale L1 loss introduced by the critic network, which guides the Scale-Att-ASPP module learning to focus on the optimal scale of the skin lesion feature. Finally, our network significantly improves the segmentation performance compared with other state-of-the-art methods, especially for the JAC and SEN scores. Besides, our proposed network works efficiently and shows robustness for different datasets.

The deep neural network model based on spatial pyramid pooling for multi-scale context information has been widely adopted in image semantic segmentation. However, this structure reduces the feature resolution during the inference process, which should be recovered by bilinear interpolation. We find that the performance of boundary location heavily depends on the interpolation operation. Assuming that the foreground object has a boundary with uneven curvature changes, it will be difficult to preserve details in the lower resolution feature map. In this case, if the model achieves near-optimal fitting, the prediction of the boundary point needs to satisfy strict constraints. But this condition is very difficult to meet for the model. For this purpose, we propose a constraint balance factor (CBF) relaxation method to improve the fitting ability of the model. The constraint balance factor is learned end-to-end by a sub-network. This sub-network can be easily embedded into a semantic segmentation network that requires interpolation without too many extra parameter increments. The experimental results illustrate that the proposed CBFNet performs well on PASCAL VOC 2012 and Cityscapes datasets.

Text detection in natural scene images is a key prerequisite for computer vision tasks such as image search, blind navigation, autopilot, and multi-language translation. Existing text detection methods only detect partial region of large-scale texts and are difficult to detect small-scale texts. Aiming at this problem, an anchor-free multi-orientation text detection method is proposed. Firstly, Feature Pyramid Network (FPN) is used to combine the multiple feature layers of Convolutional Neural Network (CNN) to predict the geometric properties of text, which can be used to expand the receptive field of each pixel and thus help to detect more large-scale texts. Secondly, a new loss function independent of the scale of text is designed, which enables the pixels in the small-scale text to have a larger calculation weight, thereby facilitating the detection of small-scale texts. Finally, the results of pixel-level semantic segmentation are used to filter obviously unreasonable candidate text boxes, and at the same time improve the accuracy and recall rate of text detection. The experimental results on ICDAR 2015 and MSRA-TD500 prove the good performance of our method.

Semantic segmentation of surgical instruments plays a critical role in computer-assisted surgery. However, specular reflection and scale variation of instruments are likely to occur in the surgical environment, undesirably altering visual features of instruments, such as color and shape. These issues make semantic segmentation of surgical instruments more challenging. In this paper, a novel network, Pyramid Attention Aggregation Network, is proposed to aggregate multi-scale attentive features for surgical instruments. It contains two critical modules: Double Attention Module and Pyramid Upsampling Module. Specifically, the Double Attention Module includes two attention blocks (i.e., position attention block and channel attention block), which model semantic dependencies between positions and channels by capturing joint semantic information and global contexts, respectively. The attentive features generated by the Double Attention Module can distinguish target regions, contributing to solving the specular reflection issue. Moreover, the Pyramid Upsampling Module extracts local details and global contexts by aggregating multi-scale attentive features. It learns the shape and size features of surgical instruments in different receptive fields and thus addresses the scale variation issue. The proposed network achieves state-of-the-art performance on various datasets. It achieves a new record of 97.10% mean IOU on Cata7. Besides, it comes first in the MICCAI EndoVis Challenge 2017 with 9.90% increase on mean IOU.

Feature pyramid is the mainstream method for multi-scale object detection. In most detectors with feature pyramid, each proposal is predicted based on feature grids pooled from only one feature level, which is assigned heuristically. Recent studies report that the feature representation extracted using this method is sub-optimal, since they ignore the valid information exists on other unselected layers of the feature pyramid. To address this issue, researchers present to fuse valid information across all feature levels. However, these methods can be further improved: the feature fusion strategies, which use common operation (element-wise max or sum) in most detectors, should be replaced by a more flexible way. In this work, a novel method called feature adaptive selection subnetwork (FAS-Net) is proposed to construct effective features for detecting objects of different scales. Particularly, its adaption consists of two level: global attention and local adaptive selection. First, we model the global context of each feature map with global attention based feature selection module (GAFSM), which can strengthen the effective features across each layer adaptively. Then we extract the features of each region of interest (RoI) on the entire feature pyramid to construct a RoI feature pyramid. Finally, the RoI feature pyramid is sent to the feature adaptive selection module (FASM) to integrate the strengthened features according to the input adaptively. Our FAS-Net can be easily extended to other two-stage object detectors with feature pyramid, and supports to analyze the importance of different feature levels for multi-scale objects quantitatively. Besides, FAS-Net can also be further applied to instance segmentation task and get consistent improvements. Experiments on PASCAL07/12 and MSCOCO17 demonstrate the effectiveness and generalization of the proposed method.

Semantic segmentation networks usually utilize special pyramid structure after encoder or combine low-level and high-level feature maps in decoder to capture multi-scale context information, which we term them feature combination and feature connection, respectively. However, both such frameworks are less valid with the fixed geometric structure or unsuitable interim. In this paper, we advocate Dynamic Attention Network (DAN) to solve these problems. First, we design a Deformable Attention Pyramid (DAP) module to perform a self-adjustable descriptor of high-level output, which utilizes deformable function to model geometric transformation. With DAP, semantic information can be captured effectively. Second, we propose a Fusing Attention Interim (FAI) module to guide the back-propagation of long-short range information in each level of decoder. We evaluate DAN on the challenging PASCAL VOC 2012 and Cityscapes segmentation benchmarks and find that it achieves state-of-the-art results without post-processing. Our observation can be concluded that the flexible structure that possesses dynamic attention mechanism is beneficial to learn multi-scale context information.

This letter presents a self-supervised framework for learning depth from monocular videos. In particular, the main contributions of this letter include: (1) We present a windowed bundle adjustment framework to train the network. Compared to most previous works that only consider constraints from consecutive frames, our framework increases the camera baseline and introduces more constraints to avoid overfitting. (2) We extend the widely used U-Net architecture by applying a Spatial Pyramid Net (SPN) and a Super Resolution Net (SRN). The SPN fuses information from an image spatial pyramid for the depth estimation, which addresses the context information attenuation problem of the original U-Net. The SRN learns to estimate a high resolution depth map from a low resolution image, which can benefit the recovery of details. (3) We adopt a clip loss function to handle moving objects and occlusions that were solved by designing complicated network or requiring extra information (such as segmentation mask [1] ) in previous works. Experimental results show that our algorithm provides state-of-the-art results on the KITTI benchmark.

The deep neural network has been widely used in semantic segmentation, especially in tumor image segmentation. The segmentation performance of traditional methods cannot meet the high standard of clinical application. In this paper, we propose a new neural network model called path aggregation U-Net (PAU-Net) model for brain tumor segmentation with multi-modality magnetic resonance imaging (MRI). Specifically, we shorten the distance between output layers and deep features by bottom-up path aggregation encoder (PA), reducing the introduction of noises. We present the enhanced decoder (ED) to reserve more intact information. The efficient feature pyramid (EFP) is used to improve mask prediction further, using fewer resources to complete the feature pyramid effect. Finally, experiments in BraTS2017 and BraTS2018 datasets are performed. The results show that the proposed method outperforms state-of-the-art methods.

This paper focuses on semantic segmentation of scenes by capturing the appropriate scale, rich detail and contextual dependencies in a feature representation. Semantic segmentation is a pixel-level classification task and has made steady progress on the basis of fully convolutional networks (FCNs). However, we find there is still room for improvement in the following aspects. The first is that the pixel itself has not enough information for semantic prediction, it needs to look around to determine which category it belongs to. However, the fixed size of the receptive field defined by the network is not suitable for all pixels when an image contains objects with various scales. The second is that the extracted scale-aware features do not handle sharper object boundaries due to low-resolution. The final aspect is regarding the ability to model long-range dependencies. In order to solve the above challenges, in this paper we propose three modules: Scale-aware spatial pyramid pool module, Encoder mask module and Scale-attention module (SSPP-ES). Extensive experiments on the Cityscapes and ADE20K benchmarks demonstrate the effectiveness of our approach for semantic segmentation.

Maxillary sinus segmentation plays an important role in the choice of therapeutic strategies for nasal disease and treatment monitoring. Difficulties in traditional approaches deal with extremely heterogeneous intensity caused by lesions, abnormal anatomy structures, and blurring boundaries of cavity. 2D and 3D deep convolutional neural networks have grown popular in medical image segmentation due to utilization of large labeled datasets to learn discriminative features. However, for 3D segmentation in medical images, 2D networks are not competent in extracting more significant spacial features, and 3D ones suffer from unbearable burden of computation, which results in great challenges to maxillary sinus segmentation. In this paper, we propose a deep neural network with an end-to-end manner to generalize a fully automatic 3D segmentation. At first, our proposed model serves a symmetrical encoder-decoder architecture for multitask of bounding box estimation and in-region 3D segmentation, which cannot reduce excessive computation requirements but eliminate false positives remarkably, promoting 3D segmentation applied in 3D convolutional neural networks. In addition, an overestimation strategy is presented to avoid overfitting phenomena in conventional multitask networks. Meanwhile, we introduce residual dense blocks to increase the depth of the proposed network and attention excitation mechanism to improve the performance of bounding box estimation, both of which bring little influence to computation cost. Especially, the structure of multilevel feature fusion in the pyramid network strengthens the ability of identification to global and local discriminative features in foreground and background achieving more advanced segmentation results. At last, to address problems of blurring boundary and class imbalance in medical images, a hybrid loss function is designed for multiple tasks. To illustrate the strength of our proposed model, we evaluated it against the state-of-the-art methods. Our model performed better significantly with an average Dice 0.947±0.031, VOE 10.23±5.29, and ASD 2.86±2.11, respectively, which denotes a promising technique with strong robust in practice.

The main challenge for scene parsing arises when complex scenes with highly diverse objects are encountered. The objects not only differ in scale and appearance but also in semantics. Previous works focus on encoding the multi-scale contextual information (via pooling or atrous convolutions) generally on top of compact high-level features (i.e., at a single stage). In this work, we argue that a rich set of cues exist at multiple stages of the network, encapsulating low, mid and high-level scene details. Therefore, an optimal scene parsing model must aggregate multi-scale context at all three levels of the feature hierarchy; a capability that lacks in state-of-the-art scene parsing models. To address this limitation, we introduce a novel architecture with three new blocks that systematically aggregate low, mid and high tier features. The heart of our approach is a high-level feature aggregation module that augments sparsely connected atrous convolution with dense local and layer-wise connections to avoid gridding artifacts. Besides, we employ a novel feature pyramid augmentation and semantic refinement unit to generate low- and mid-level features that are mixed with high-level features at the decoder. We extensively evaluate our proposed approach on the large-scale Cityscapes and ADE2K benchmarks. Our approach surpasses many latest models on both datasets, achieving mean intersection-over-union (mIoU) scores of 80.5% and 44.0% on Cityscapes and ADE20K, respectively.

The semantic segmentation of remote sensing (RS) image is a hot research field. With the development of deep learning, the semantic segmentation based on a full convolution neural network greatly improves the segmentation accuracy. The amount of information on the RS image is very large, but the sample size is extremely uneven. Therefore, even the common network can segment RS images to a certain extent, but the segmentation accuracy can still be greatly improved. The common neural network deepens the network to improve the classification accuracy, but it has a lot of loss to the target spatial features and scale features, and the existing common feature fusion methods can only solve some problems. A segmentation network is built to solve the above problems very well. The network employs the InceptionV-4 network as the backbone and improves it. We modify the network structure and introduce the changed Atrous Spatial Pyramid Pooling module to extract the multi-scale features of the target from different training stages. Without losing the depth of the network, using Inception blocks to strengthen the width of the network can obtain more abstract features. At the same time, the backbone network is used for semantic fusion of the context, it can retain more spatial features, then an effective decoder network is designed. Finally, evaluate our model on the ISPRS 2D Semantic Labeling Contest Potsdam and Inria Aerial Image Labeling Dataset. The results show that the network has very superior performance, reaching 89.62% IOU score and 94.49% F1 score on the Potsdam dataset, and the IOU score on the Inria dataset has been greatly improved.

Finding and segmenting objects in noisy low-contrast in situ underwater plankton images is challenging because of the difficulty of separating potential plankton objects from the complex background and numerous and diverse other particles. In the present study, a dynamic downscaling model was developed to rapidly extract complete and clean regions of interest (ROIs) from images with highly variable content and quality. The original image was downscaled, and dynamic segmentation was performed in a scale pyramid space to ensure the integrity of weak targets based on local two-dimensional (2D) entropy parameters. Subsequently, a series of local thresholds and clustering gradients was examined iteratively for ROI selection. The performance of the local 2D entropy parameters relative to water turbidity (in a scattering medium and with high background noise) and image size was examined. To suppress the background and increase the sharpness of potential targets, a sharpness descriptor and gradient clustering were employed. The method was compared with the currently commonly used local threshold-based Sauvola segmentation using the same set of images. The results showed that the proposed method improves the ROI extraction accuracy and reduces oversegmentation for in situ underwater plankton images. It was concluded that the proposed method is a fast and robust segmentation technique and could facilitate the deployment of in situ plankton imaging systems for process-based research and routine plankton monitoring.

Traffic congestion due to vehicular accidents seriously affects normal travel, and accurate and effective mitigating measures and methods must be studied. To resolve traffic accident compensation problems quickly, a vehicle-damage-detection segmentation algorithm based on transfer learning and an improved mask regional convolutional neural network (Mask RCNN) is proposed in this paper. The experiment first collects car damage pictures for preprocessing and uses Labelme to make data set labels, which are divided into training sets and test sets. The residual network (ResNet) is optimized, and feature extraction is performed in combination with Feature Pyramid Network (FPN). Then, the proportion and threshold of the Anchor in the region proposal network (RPN) are adjusted. The spatial information of the feature map is preserved by bilinear interpolation in ROIAlign, and different weights are introduced in the loss function for different-scale targets. Finally, the results of self-made dedicated dataset training and testing show that the improved Mask RCNN has better Average Precision (AP) value, detection accuracy and masking accuracy, and improves the efficiency of solving traffic accident compensation problems.