Search

Traffic sign detection has made great progress with the rise of deep learning in recent years. As a result of the complex and changeable traffic environment, detecting small traffic signs in a real-world scene is still a challenging problem. In this paper, a novel group multi-scale attention pyramid network is proposed to address the problem. Specifically, to aggregate the feature at different scales and suppress the messy information in the background, an effective multi-scale attention module is proposed. Furthermore, a feature fusion module, named adaptive pyramid convolution, is further designed, which can drive the network to learn the optimal feature fusion pattern and construct an informative feature pyramid for detecting traffic signs in different sizes. Extensive experimental results on the public traffic sign detection datasets demonstrate the effectiveness and superiority of the proposed method when compared with several state-of-the-art methods.

This study reviews available guidelines and assessment methods for daylight performance for office buildings in Egypt to propose a Unified design guideline (UDG). Relevant literature studies are analysed, e.g. the local code for ‘improving energy efficiency for commercial buildings’ as well as a number of green building rating systems such as Leadership in Energy and Environmental Design (LEED), and The Green Pyramid Rating system. The UDG was tested for two local case studies using building simulation; a non-LEED building complying with a set of limited guiding principles of the local code, and a LEED-certified building with an additional set of guidelines, and compliance path using building simulation. The comparison showed that the integration of both prescriptive and performance-based guidelines and measurement criteria is necessary to develop the UDG tailored to address the particular requirements of office buildings in terms of daylight performance in similar climates.

In this paper, we have considered a recently reported 2-layer non-DHT-based structured P2P network. Residue Class based on modular arithmetic has been used to realize the overlay topology. At the heart of the architecture (layer-1), there exists a tree like structure, known as pyramid tree. It is not a conventional tree. A node i in this tree represents the cluster-head of a cluster of peers which are interested in a particular resource of type Ri (i.e. peers with a common interest). The cluster-head is the first among these peers to join the system. Root of the tree is assumed to be at level 1. Such a tree is a complete one if at each level j, there are j number of nodes. It is an incomplete one if only at its leaf level, say k, there are less than k number of nodes. Layer 2 consists of the different clusters. The network has some unique structural properties, e.g. each cluster has a diameter of only 1 overlay hop and the diameter of the network is just (2+2d); d being the number of levels of the layer-1 pyramid tree and d depends only on the number of distinct resources. Therefore, the diameter of the network is independent of the number of peers in the whole network. In the present work, we have used some such properties to design low latency intra and inter cluster data lookup protocols. Our choice of considering non-DHT and interest-based overlay networks is justified by the following facts: 1) intra-cluster data lookup protocol has constant complexity and complexity of inter-cluster data lookup is O(d) if tree traversal is used and 2) search latency is independent of the total number of peers present in the overlay network unlike any structured DHT-based network (as a matter fact unlike any existing P2P network, structured or unstructured). Experimental results as well show superiority of the proposed protocols to some noted structured networks from the viewpoints of search latency and complexity involved in it. In addition, we have presented in detail the process of handling churns and proposed a simple yet very effective technique related to cluster partitioning, which, in turn, helps in reducing the number of messages required to be exchanged to handle churns.

Semantic segmentation is a valuable yet challenging research in the Internet of Things (IoT), especially for some real-time and resource-constrained applications. Recently we witness a strong tendency of fusing multi-level features or multi-scale context information to achieve promising segmentation performance . However, we find that existing literature has at least one of the following issues: 1) relying on more resource-consuming feature extraction operations, e.g., standard convolution with large kernels, for multiple information fusion; 2) seldom considering that how to narrow the semantic gap between multi-layer features, leading to sub-optimal performance. To tackle these issues, we propose a novel IoT-perceptive Dual Feature Fusion Network (DFFNet) for semantic segmentation, which aims to leverage multi-level features and multi-scale context information in an efficient yet effective manner. Specifically, the multi-level feature fusion module (MFFM), which enhances the semantic consistency between multi-level features by two attention refinement blocks, is proposed to exploit multi-layer features for jointly learning spatial and semantic information with small overheads. Moreover, a multi-scale component termed as lightweight semantic pyramid module (LSPM) is presented to improve the efficiency of context encoding by depthwise factorized convolution operations . Extensive experimental results on benchmarks have demonstrated that DFFNet achieves better performance than existing advanced methods.

In existing state-of-the-art object detectors, feature pyramid networks (FPN) and multiscale feature fusion are still typically used. The traditional FPN fusion strategy is based on the top-down fusion of high-level semantic information. The top-down fusion method generally uses upsampling based on interpolation, which often results in jagged edges, mosaic distortion, and edge blurring. Moreover, in order to improve accuracy, the FPN-based fusion strategy must add multiple top-down components for fusion, which increases computational costs and leads to a poor balance between precision and speed. In this paper, we propose a novel fusion strategy based on a backbone network. We aim to design simple and efficient components for high-quality object detection. Our proposed strategy, bi-directional skip connection FPN (BiSCFPN), consists of three components: a bi-directional skip connection (BiSC), a selective dilated convolution module (SDCM), and sub-pixel convolution (SP). The BiSC aims to enhance semantic information between different feature layers in the backbone network and simultaneously uses the SDCM to improve the receptive fields of differently sized targets in the fusion stage. Finally, SP learns the relationship between the features of upsampling and downsampling images to effectively mitigate the problems caused by the traditional interpolation method. BiSCFPN achieves an average precision of 38.2% in tests with the Microsoft Common Objects in Context (MS COCO) test-dev dataset at a real-time speed of ~ 50 FPS ( 608 × 608 ) using an Nvidia GeForce RTX 2080 Ti graphics card and significantly improves the balance between precision and speed.

Learning-based methods have attracted considerable interest in image dehazing. However, most existing methods are not well adapted to different hazy conditions, especially when dealing with the heavily hazy scene. There is often a significant amount of haze that remains in the images recovered by most methods. To address this issue, we propose an end-to-end Haze Concentration Adaptive Network (HCAN), including a pyramid feature extractor (PFE), a feature enhancement module (FEM), and a multi-scale feature attention module (MSFAM) for image dehazing. Specifically, PFE based on the feature pyramid structure leverages complementary features from different CNN layers to help the clear image prediction. Then, FEM fuses four kinds of images with different haze density (i.e., three recovered images in the FEM with light haze density, and the input hazy image with strong haze condition) to guide the network to adaptively perceive images under different haze conditions. Finally, MSFAM is designed under two principles, multi-scale structure and attention mechanism. It is used to help the network produce a clear image with more details, and ease the network training. Comprehensive experiments demonstrate that the proposed HCAN performs favorably against the state-of-the-art methods in terms of PSNR, SSIM, and visual effect. The results, per-trained models and code are available at https://github.com/TaoWangzj/HCAN .

By adjusting the KOH/H2O texturing condition intendedly, different random pyramidal textures with the average pyramid size of 8 μm (large), 4 μm (medium) and 1.5 μm (small) were prepared on N type M2 monocrystalline silicon substrates for the fabrication of silicon heterojunction (SHJ) solar cell. It was evidenced that the pyramid morphology not only impacted the reflection of incident light and the heterojunction interface passivation quality but also affected the interface contact property between transparent conductive oxide (TCO) layer and Ag electrode. The texture with small pyramids presented the best anti-reflective and passivation performance due to the pyramidal size distribution uniformity, which brought the SHJ solar cell the highest short-circuit current density (JSC) and open-circuit voltage (VOC). However, the texture with small pyramids also led to the worst fill factor (FF). As a result, there was no obvious difference on the conversion efficiency for the textures with different pyramidal size. By characterizing the TCO/Ag contact, the FF reduction induced by the small pyramid texture could be attributed to the increased specific contact resistance of the TCO/Ag contact, which resulted from the fact that there were lots of voids between the TCO layer and Ag electrode, since Ag particles with a larger size than the small pyramids could not fill into the limited space between the adjacent pyramids. As a result, in order to realize the potential to enhance the SHJ solar cell conversion efficiency by adopting the texture with small pyramids, new Ag paste with smaller Ag particles should be adopted.

An increasing number of change detection models are designed with different convolutional neural network (CNNs). However, the mechanism for designing network layers that can effectively extract robust features for different scenes remains unclear. Thus, novel networks with fluid pyramid integration network (FPIN) to detect changes are proposed in this study. Specifically, we first extract multi-scale deep features from feature extraction network. Higher layers extract semantic features robust to illumination and camera pose variations, while lower layers extract texture features that generate clear boundaries and changes with small scales. To employ the advantages of the features extracted from different layers, FPIN progressively fuses higher and lower layer features in a hierarchical order. We propose three different change detection networks based on FPIN and validate them on three publicly available change detection benchmark datasets. Experimental results showed that the proposed networks are better than state-of-the-art change detection methods.

Unmanned aerial vehicles, or drones, are poised to solve many problems associated with data collection in complex urban environments. Drones are easy to deploy, have a great ability to move and explore the environment, and are relatively cheaper than other data collection methods. This study investigated the use of Cascade Region-based convolutional neural network (R-CNN) networks to enable automatic vehicle counting and tracking in aerial video streams.The presented technique combines feature pyramid networks and a Cascade R-CNN architecture to enable accurate detection and classification of vehicles.The paper discusses the implementation and evaluation of the detection and tracking techniques and highlights their advantages when they are used to collect traffic data.

RoIPool/RoIAlign is an indispensable process for the typical two-stage object detection algorithm, it is used to rescale the object proposal cropped from the feature pyramid to generate a fixed size feature map. However, these cropped feature maps of local receptive fields will heavily lose global context information. To tackle this problem, we propose a novel end-to-end trainable framework, called global context aware (GCA) RCNN, aiming at assisting the neural network in strengthening the spatial correlation between the background and the foreground by fusing global context information. The core component of our GCA framework is a context aware mechanism, in which both global feature pyramid and attention strategies are used for feature extraction and feature refinement, respectively. Specifically, we leverage the dense connection to improve the information flow of the global context at different stages in the top-down process of FPN, and further use the attention mechanism to refine the global context at each level in the feature pyramid. In the end, we also present a lightweight version of our method, which only slightly increases model complexity and computational burden. Experimental results on COCO benchmark dataset demonstrate the significant advantages of our approach.

In this work, a square pyramid solar still is designed, fabricated and tested during recommended days at the four seasons of Al Kufa climatic conditions (elevation is 30 m and location is 32° North and 44° East), Iraq. The proposed design of the still can be used at any remote location in Iraq where is no electricity and high solar intensity. Designing the affordable solar still was mainly due to simplicity in its operation process, long life and low cost. Data obtained from an advanced simulation modeling of multi-shaped solar stills are used to determine the temperature gradient and the heat flux distributions during unsteady-state conditions. The procedure of the modelling is included: volumetric properties of the solid and fluid regions, thermal loads on the still, mesh generation, grid convergence index and grid refinery processes. The experiments were measured during the four seasons to investigate the temperatures of water, glass, ambient, water production and solar intensity. The results show that the maximum daily water production of the designed still is 2.2 L/m2 in 17th of July with a low cost of 15 US dollars. The numerical model is predicted the thermal behavior of triangle, pyramid and pentagon types of solar stills. The results also show that the efficiency of the designed still is significantly improved to 60% compared with other experimental solar stills. The results also demonstrate that the manufacturing and transferring of the portable solar still during 20 years has a reasonable effect on the environment in terms of air acidification and water eutrophication.

Many deep learning detectors have been proposed to address the scale variation issue in pedestrian detection. However, it is still far from being fully solved because most of these detectors handle the issue by directly fusing multiple extracted features with the weighted average or concatenation. Therefore, this paper designs a novel and effective anchor-free network (namely MPAF-Net) for multi-scale pedestrian detection. Specifically, we first improve Res2Net as our backbone network by designing a novel bottleneck block to make the proposed detector has a strong multi-scale representation ability for pedestrians. Furthermore, based on Conditional Random Fields (CRFs), the multi-scale features obtained from three different feature pyramid levels of backbone network are refined with a designed message passing mechanism. By doing so, the complementary information from the features at other scales can be dynamically passed to enhance the scale-specific feature. Finally, pedestrian detection is simplified as a straightforward center and scale prediction task by performing convolution on the concatenation of refined multi-scale convolution features. This way, the proposed MPAF-Net can be trained in an end-to-end fashion. Experimental results show that MPAF-Net achieves state-of-the-art performance on CityPersons, Caltech, and CrowdHuman benchmarks.

In this paper, we propose a novel multi-scale fusion framework for low-illumination image enhancement, which effectively enhances images taken under various low-light conditions. Based on the high dynamic range imaging technique, we first employ a novel remapping function to generate a sequence of artificial multi-exposure images. The generated sequence of images ensures that the contrast of each intensity interval of the input image is enhanced at least once. Then three fusion-relevant features, namely, exposure, global contrast and local contrast, are selected as the weight maps. Combined with the weight maps, a pyramid fusion scheme is introduced to do a layer-by-layer integration of the different frequency bands of the image layer by layer. In addition, a strategy for extracting details from the original image is designed, which effectively maintains the detail information without causing colour distortions. The framework is very efficient and suitable for mobile devices because most of the calculations are at the pixel-level. Extensive experiments have shown that the proposed approach yields comparable and better performances in comparisons with the state-of-the-art competing techniques in both qualitative and quantitative evaluations.

Proposal-free instance segmentation methods mainly generate instance-agnostic semantic segmentation labels and instance-aware features to group pixels into different object instances. However, previous methods mostly employ separate modules for these two sub-tasks and require multiple passes for inference. In addition to the lack of efficiency, previous methods also failed to perform as well as proposal-based approaches. To this end, this work proposes a single-shot proposal-free instance segmentation method that requires only one single pass for prediction. Our method is based on learning an affinity pyramid, which computes the probability that two pixels belong to the same instance in a hierarchical manner. Moreover, incorporating with the learned affinity pyramid, a novel cascaded graph partition (CGP) module is presented to fuse the two predictions and segment instances efficiently. As an additional contribution, we conduct an experiment to demonstrate the benefits of proposal-free methods in capturing detailed structures from finely annotated training examples. Our approach is evaluated on the Cityscapes and COCO datasets and achieves state-of-the-art performance.

In this paper, we explore an issue that is to detect vehicle headlights from the nighttime traffic surveillance images with highly reflections. In the night, reflections on the road (water) surface, vehicles bodies, or some reflective objects (such as lane markings, traffic signs) will interfere the headlight detection seriously. Although the existing methods have achieved good results, however, most of them failed to detect the headlight when headlights are far from camera. In order to solve the issue, we propose a novel method for vehicle headlights detection. The proposed method makes full use of the brightness and gradient information of the headlights in the night. First, we propose an effective region-of-interest (ROI) segmentation method which is based on multi-scale local saliency detection. The method pre-serve faint or small-sized objects and retain the original shape of the object to the greatest extent. Then, we compute the pyramid histogram of oriented gradients (PHOG) features, which are used to train support vector machine (SVM) classifier. Finally, the extracted bright blocks are classified according to the pre-trained SVM classifier. Experimental results and quantitative evaluations in different scenes demonstrate that our proposed method can achieve a better result compared with previous methods.

Convolutional Neural Networks (CNNs) have played an important role in saliency detection. How to detect a salient object as a whole is a key issue. However, most existing learning-based methods are not accurate enough to detect salient objects in complex scenes, such as easily overlooked small salient areas in a whole salient object, which is called scale imbalance problem in this paper. To address this issue, Scale Balance Network (SBN) based on fully convolutional network is proposed to accurately recognize and comprehensively detect salient objects. Firstly, to detect more small salient areas, a specially designed backbone instead of common backbone is adopted in this paper, which can capture larger resolution with more spatial features in deeper layers. Secondly, we present a novel progressive pyramid mechanism named Connective Feature Pyramid Module (CFPM), aiming to make the network focus on the balance between the large salient areas and the small ones. Finally, we present an Edge Enhancement Architecture with Various Kernels (EEAVK) to locate the saliency maps and refine the boundary features. Experimental results on five benchmark datasets show that the proposed SBN method achieves consistently superior performance in comparison with other state-of-the-art ones under different evaluation metrics.

Feature pyramids have delivered significant improvement in object detection. However, building effective feature pyramids heavily relies on expert knowledge, and also requires strenuous efforts to balance effectiveness and efficiency. Automatic search methods, such as NAS-FPN, automates the design of feature pyramids, but the low search efficiency makes it difficult to apply in a large search space. In this paper, we propose a novel search framework for a feature pyramid network, called AutoDet, which enables to automatic discovery of informative connections between multi-scale features and configure detection architectures with both high efficiency and state-of-the-art performance. In AutoDet, a new search space is specifically designed for feature pyramids in object detectors, which is more general than NAS-FPN. Furthermore, the architecture search process is formulated as a combinatorial optimization problem and solved by a Simulated Annealing-based Network Architecture Search method (SA-NAS). Compared with existing NAS methods, AutoDet ensures a dramatic reduction in search times. For example, our SA-NAS can be up to 30x faster than reinforcement learning-based approaches. Furthermore, AutoDet is compatible with both one-stage and two-stage structures with all kinds of backbone networks. We demonstrate the effectiveness of AutoDet with outperforming single-model results on the COCO dataset. Without pre-training on OpenImages, AutoDet with the ResNet-101 backbone achieves an AP of 39.7 and 47.3 for one-stage and two-stage architectures, respectively, which surpass current state-of-the-art methods.

Recently, object detection has been a vital aspect in the vision community, while scale variation of objects in images or videos usually brings challenge for performance improvement. To combat this problem, conventional paradigms generally adopt image pyramid or Feature Pyramid Network (FPN) to process objects at different scales. However, existing multi-scale deep convolution neural networks mostly set different scales in a heuristic way, which may introduce inconsistency between the region of interest and the semantic scope. In this paper, we propose an innovative paradigm called Consistent Scale Normalization (CSN) to weaken the influence of scale variation for object detection. The proposed CSN can realize a consistent compression for the scale space of objects, in both training and testing phases. Extensive experimental testing is performed on COCO object detection benchmark in comparison with several state-of-the-art methods. In addition to object detection, experiments on instance segmentation and multi-task human pose estimation are also conducted. Furthermore, the CSN paradigm is beneficial to reduce the difficulty of network learning. The results verify the effectiveness and superiority of the CSN paradigm.

High spatial resolution (HSR) imagery scene classification has become a hot research topic in remote sensing. Scene classification method based on the handcrafted features, such as the bag-of-visual-words (BoVW) model, describes an image by extracting local features of the scene and mapping them to the dictionary space, but usually uses a shallow structure and loses the spatial distribution characteristics of the scene. The method based on deep learning extracts hierarchical features to describe the scene, which can maintain the spatial position information well. However, deep features in different levels have scale recognition restrictions for multi-scale ground objects, and cannot understand complex scenes well. In this paper, the multi-level convolutional pyramid semantic fusion (MCPSF) framework is proposed for HSR imagery scene classification. Differing from previous scene classification methods, which integrate the feature of different levels directly, of which the fusion features have large differences in both sparsity and eigenvalue magnitude, MCPSF integrates multi-level semantic features extracted by BoVW model and convolutional neural network (CNN) model. In MCPSF, two convolution pyramid feature expression strategies are proposed to enhance the ability of capturing multi-scale land objects, i.e., local and convolutional pyramid based BoVW (LCPB) model and local and convolutional pyramid based pooling-stretched (LCPP) model. The effectiveness of the proposed method is verified on 21-class UC Merced (UCM) dataset and 30-class Aerial Image Dataset (AID). The framework was also transferred toa case study of scene annotation in Wuhan. The proposed framework significantly improves the performance when compared with other state-of-the-art methods.

Semantic segmentation is a task that covers most of the perception needs of intelligent vehicles in an unified way. Recent studies witnessed that attention mechanisms achieve impressive performance in computer vision task. Current attention mechanisms based segmentation methods differ with each other in position and form of the attention mechanism, and perform differently in practice. This paper firstly introduces the effectiveness of multi-scale context features and attention mechanisms in segmentation tasks. We find that multi-scale and channel attention can play a vital role in constructing effective context features. Based on this analysis, this paper proposes an efficient attention pyramid network (EAPNet) for semantic segmentation. Specifically, to efficient handle the problem of segmenting objects at multiple scales, we design efficient channel attention pyramid (ECAP) which employ atrous convolution with channel attention in cascade or in parallel to capture multi-scale context by using multiple atrous rates. Furthermore, we propose a residual attention fusion block (RAFB), whose purpose is to simultaneously focus on meaningful low-level feature maps and spatial location information. At the same time, we will explore different channel attention modules and spatial attention modules, and describe their impact on network performance. We empirically evaluate our EAPNet on two semantic segmentation datasets, including PASCAL VOC 2012 and Cityscapes datasets. Experimental results show that without MS COCO pre-training and any post-processing, EAPNet achieved 81.7% mIoU on the PASCAL VOC 2012 validation set. With deeplabv3+ as the benchmark, EAPNet improve the model performance of more than 1.50% mIoU.

Disease is one of the main factors affecting crop growth. How to reflect the external morphological features of the disease and completely retain the color and texture information of the disease area is one of the key research issues for crop disease segmentation. Meanwhile, aiming at the problem of low segmentation accuracy with traditional convolution neural network-based methods in the crop disease leaf image, this paper proposes a spatial pyramid-oriented encoder-decoder cascade convolution neural network for crop disease leaf segmentation. The network consists of a region disease detection network and a region disease segmentation network. Region disease detection network is a kind of network combining cascade convolution neural network with spatial pyramid. This method connects the three-level convolution neural network model, where the structure of the three-level neural network model varies from simple to complex. Different crop disease leaf features are extracted from the different neural network levels. And images are screened to complete the detection of crop disease leaf. What’s more, a space pyramid pooling layer is added to each network level. This pooling strategy does not require fixed size input, which increases the size selection of input model. The region segmentation network is established based on the Encoder-Decoder structure. The multi-scale convolution kernel is used to improve the local receptive field of the original convolution kernel and accurately segment the crop disease leaf area. Finally, we conduct experiments on the crop disease leaf images under different conditions, the results show that the proposed method has higher segmentation accuracy. In terms of Precision, Correct segmentation, over-segmentation and under-segmentation indexes, etc., the average values of proposed method are more than 90%. The average dice similarity coefficient is over 95% under different background. Moreover, it can meticulously reflect the external morphological features of the crop disease leaf and relatively better retain the color and texture information.

Automatic surface defect inspection (ASDI) is a crucial and challenging problem in industry because it affects the quality and efficiency of production greatly. Deep learning-based methods achieve promising improvements for surface detection, but they rely on a massive training data set that is impractical in industry. In this study, we propose a generic method that works effectively even on the small size training data set. Especially, we introduce a ResMask generative adversarial network (GAN) framework that is a residual GAN to expand the insufficient defect data sets. Meanwhile, the existing inspection data sets are so much easier to detect than the data sets in the real industrial scenarios that new industrial surface defect data sets containing more diverse and challenging images are established. Then, a coarse-to-fine module (CFM) that consists of a coarse detection subnetwork and a fine segmentation subnetwork is proposed for the needs of fast detection for high-resolution images. In the coarse detection stage, spatial pyramid pooling (SPP) is utilized to increase the receptive field of the network, reduce the false detection rate, and determine the approximate location of defects. In the fine segmentation stage, the receptive field of the network is enlarged by atrous SPP (ASPP), and skip links that incorporate low-level with high-level features achieve pixelwise precision on defect segmentation. Finally, our algorithm has achieved state-of-the-art results in DAGM, HR, and WB data sets (0.859, 0.761, and 0.805, respectively) according to MIoU. It achieves an average processing time of 44.4 ms on the test images with a resolution of $512 \,\,\times \,\, 512$ and 131.6 ms for $2048 \times 2048$ images. On the DAGM data set, the detection accuracy of mean intersection over union (MIoU) reaches 0.859.

Fruit detection plays a vital role in harvesting robot platforms. However, complicated environment attributes such as illumination variation, occlusion, have made fruit detection a challenging task. A robust YOLOMuskmelon model that is accurate and fast was proposed to solve detection difficulties. The YOLOMuskmelon model incorporated ReLU activated ResNet43 backbone with new 2,3,4,3,2 residual block arrangement, spatial pyramid pooling(SPP), complete Intersection over Union (CIoU) loss, feature pyramid network(FPN), and distance Intersection over Union-Non Maximum Suppression(DIoU-NMS) to improve detection performance. The obtained average precision (AP) results of YOLOMuskmelon at 89.6% is greater than YOLOv3 at 82.3%, YOLOResNet50 at 85.5%, but less than YOLOv4 at 91.6%. However, the detection speed of YOLOMuskmelon at 96.3 frame per second(fps) outperformed YOLOv3 at 56.6fps, YOLOv4 at 54.1fps and YOLOResNet50 at 71.2fps. Meanwhile, the YOLOMuskmelon which is 56.1% faster than YOLOv4 model showed a better generalization and real-time fruit harvesting robots prospect.

The convolutional neural network (CNN)-based multi-focus image fusion methods which learn the focus map from the source images have greatly enhanced fusion performance compared with the traditional methods. However, these methods have not yet reached a satisfactory fusion result, since the convolution operation pays too much attention on the local region and generating the focus map as a local classification (classify each pixel into focus or de-focus classes) problem. In this article, a global-feature encoding U-Net (GEU-Net) is proposed for multi-focus image fusion. In the proposed GEU-Net, the U-Net network is employed for treating the generation of focus map as a global two-class segmentation task, which segments the focused and defocused regions from a global view. For improving the global feature encoding capabilities of U-Net, the global feature pyramid extraction module (GFPE) and global attention connection upsample module (GACU) are introduced to effectively extract and utilize the global semantic and edge information. The perceptual loss is added to the loss function, and a large-scale dataset is constructed for boosting the performance of GEU-Net. Experimental results show that the proposed GEU-Net can achieve superior fusion performance than some state-of-the-art methods in both human visual quality, objective assessment and network complexity.

The performance of state-of-the-art object skeleton detection (OSD) methods have been greatly boosted by Convolutional Neural Networks (CNNs). However, the most existing CNN-based OSD methods rely on a ‘skip-layer’ structure where low-level and high-level features are combined to gather multi-level contextual information. Unfortunately, as shallow features tend to be noisy and lack semantic knowledge, they will cause errors and inaccuracy. Therefore, in order to improve the accuracy of object skeleton detection, we propose a novel network architecture, the Multi-Scale Bidirectional Fully Convolutional Network (MSB-FCN), to better gather and enhance multi-scale high-level contextual information. The advantage is that only deep features are used to construct multi-scale feature representations along with a bidirectional structure for better capturing contextual knowledge. This enables the proposed MSB-FCN to learn semantic-level information from different sub-regions. Moreover, we introduce dense connections into the bidirectional structure to ensure that the learning process at each scale can directly encode information from all other scales. An attention pyramid is also integrated into our MSB-FCN to dynamically control information propagation and reduce unreliable features. Extensive experiments on various benchmarks demonstrate that the proposed MSB-FCN achieves significant improvements over the state-of-the-art algorithms.

Expression recognition in the wild is easily distorted by nonfrontal and asymmetry faces. In nonfrontal faces, some areas are compressed and distorted. Even after frontalization, these compressed areas may still be blurred and distort expression recognition. Additionally, asymmetrical expressions are common on half or local face areas and produce incorrect expression features. Therefore, this paper presents a half-face frontalization and pyramid Fourier frequency conversion method. Despite the location, range and intensity of incorrect expressions in nonfrontal faces being unknown, according to discrete Fourier transform, it can be proven that the frequency band of the correct expression is much larger than that of incorrect expression on the same face. This can be taken advantage of by pyramid frequency conversion, which is designed based on Fourier frequency conversion. It can adjust incorrect expression frequency in multiscales to take them out off the band-pass of the convolution operations of deep learning and be eliminated completely, whereas correct expression information is reserved. Thus, expressions can be recognized effectively.

Existing human pose estimation methods usually have a high computational load, which is very unfavorable for resource-limited equipment. To address this issue, we propose a low computational-cost deep supervision pyramid network called DSPNet. Firstly, we design a lightweight up-sampling unit instead of transposed convolution as a decoder for the network. In the case of decreased computation, it has brought an increase in prediction accuracy. Secondly, we present a novel deep supervision pyramid architecture to improve the multi-scale obtaining ability of MSRA SimpleBaseline while not bringing any increase in the number of parameters. The experimental results on both MPII and COCO pose estimation benchmarks illustrate that DSPNet achieves almost equivalent state-of-the-art results with a low computational load. Especially, the computational cost of DSPNet is 12.7% of SimpleBaseline and the estimation accuracy is improved by 0.9 points when both methods use the same backbone network (EfficientNet) on MPII validation set. The code of the proposed method is availabe at https://github.com/sumaliqinghua/DSPNet .

Due to the advantages of real-time detection and improved performance, single-shot detectors have gained great attention recently. To solve the complex scale variations, single-shot detectors make scale-aware predictions based on multiple pyramid layers. Typically, small objects are detected on shallow layers while large objects are detected on deep layers. However, the features in the pyramid are not scale-aware enough, which limits the detection performance. Two common problems in single-shot detectors caused by object scale variations can be observed: (1) false negative problem, i.e. , small objects are easily missed due to the weak features; (2) part-false positive problem, i.e. , the salient part of a large object is sometimes detected as an object. With this observation, a new Neighbor Erasing and Transferring (NET) mechanism is proposed for feature scale-unmixing to explore scale-aware features in this paper. In NET, a Neighbor Erasing Module (NEM) is designed to erase the salient features of large objects and emphasize the features of small objects in shallow layers. A Neighbor Transferring Module (NTM) is introduced to transfer the erased features and highlight large objects in deep layers. With this mechanism, a single-shot network called NETNet is constructed for scale-aware object detection. In addition, we propose to aggregate nearest neighboring pyramid features to enhance our NET. Experiments on MS COCO dataset and UAVDT dataset demonstrate the effectiveness of our method. NETNet obtains 38.5% AP at a speed of 27 FPS and 32.0% AP at a speed of 55 FPS on MS COCO dataset. As a result, NETNet achieves a better trade-off for real-time and accurate object detection.

Judging imaging quality is an important part of the maintenance of visual intelligent monitoring systems for electrical power scenes. However, accurate and efficient identification of possible abnormalities in imaging quality remains challenging. This paper proposes a novel multi-region two-stream deep architecture to improve judging abnormalities. The proposed architecture incorporates two-stream scheme and multi-region strategy to identify relevant information and explore hidden details. More specifically, in addition to color and intensity in the original images, the two-stream scheme uses high-frequency structure information from gradient images to enhance its performance. The multi-region strategy employs spatial pyramid random cropping and region fusion to handle locally non-uniform changes among categories: spatial pyramid random cropping characterizes images at different spatial pyramid levels, while region fusion focuses attention on cropped regions relevant to quality perception by using adaptive learning weights in a fully connected layer. In this way, the proposed strategy guides the framework to adequately and adaptively explore the discriminative regions hidden in the input images, and provides an end-to-end learning procedure. Experimental results demonstrate its strong performance for judging abnormalities, and the proposed method can be easily extended to the entire surveillance system.

Objects at different spatial positions in an image exhibit different scales. Adaptive receptive fields are expected to capture suitable ranges of context for accurate pixel level semantic prediction. Recently, atrous convolution with different dilation rates has been used to generate features of multi-scales through several branches which are then fused for prediction. However, there is a lack of explicit interaction among the branches of different scales to adaptively make full use of the contexts. In this paper, we propose a Content-Adaptive Scale Interaction Network (CASINet) to exploit the multi-scale features for scene parsing. We build CASINet based on the classic Atrous Spatial Pyramid Pooling (ASPP) module, followed by a proposed contextual scale interaction (CSI) module, and a scale adaptation (SA) module. Specifically, in the CSI module, for each spatial position of some scale, instead of being limited by a fixed set of convolutional filters that are shared across different spatial positions for feature learning, we promote the adaptivity of the convolutional filters to spatial positions. We achieve this by the context interaction among the features of different scales. The SA module explicitly and softly selects the suitable scale for each spatial position and each channel. Ablation studies demonstrate the effectiveness of the proposed modules. We achieve state-of-the-art performance on three scene parsing benchmarks Cityscapes, ADE20K and LIP.

Scene text detection methods have achieved significant progresses. However, stack-omnidirectional text dilemma, under-segmentation of very close text words, and over-segmentation of arbitrary-shape long text lines, are still main challenges. Motivated by these problems, we proposed a two stage method called omnidirectional pyramid mask proposal text detector (OPMP). OPMP removes anchor mechanism that requires heuristic non-maximum suppress processing. Instead, it uses an effective pyramid lengthwise and sidewise residual sequence modeling method to produce arbitrary-shape proposals. To accurately extract the features of text shape, OPMP enhances the backbone layers by a multiple arbitrary-shape fitting mechanism. Finally, a multi-grain text classification module is proposed, which reclassifies each text region robustly. Comprehensive ablation studies demonstrate the effectiveness of each proposed component. In addition, experiments on various benchmarks, including ICDAR2015, MLT, MSRA-TD500, CTW1500, and Total-text, show that our method outperforms previous state-of-the-art methods.

Small fasteners are widely used in a freight train. However, due to the complex working environment, the missing of fasteners always happens, causing traffic accidents and property loss. Thus, detection of the missing of small fasteners is essential for the safety of freight trains. This article takes three critical missing small fasteners as examples, pin-missing, bolt-missing, and rivet-missing defects, and proposes a two-stage small defect detection model. First, a copy–pasting method is used to augment the defects in the training image, then the ResNet-101 backbone is adopted to extract defect features, and a feature pyramid network is used to construct a feature pyramid. After obtaining defect features, an improved region proposal network is introduced to generate defect proposals. Finally, a fully convolutional neural network is trained to operate pixelwise segmentation. The experimental results on manually collected fastener defect data set show that our model possesses high precision and efficiency on the inspection of pin-missing, bolt-missing, and rivet-missing defects. Besides, the proposed model could be used for similar small defects detection.

Video frame interpolation aims to improve users' watching experiences by generating high-frame-rate videos from low-frame-rate ones. Existing approaches typically focus on synthesizing intermediate frames using high-quality reference images. However, the captured reference frames may suffer from inevitable spatial degradations such as motion blur, sensor noise, etc. Few studies have approached the joint video enhancement problem, namely synthesizing high-frame-rate and high-quality results from low-frame-rate degraded inputs. In this paper, we propose a unified optimization framework for video frame interpolation with spatial degradations. Specifically, we develop a frame interpolation module with a pyramid structure to cyclically synthesize high-quality intermediate frames. The pyramid module features adjustable spatial receptive field and temporal scope, thus contributing to controllable computational complexity and restoration ability. Besides, we propose an inter-pyramid recurrent module to connect sequential models to exploit the temporal relationship. The pyramid module integrates the recurrent module, thus can iteratively synthesize temporally smooth results. And the pyramid modules share weights across iterations, thus it does not expand the model's parameter size. Our model can be generalized to several applications such as up-converting the frame rate of videos with motion blur, reducing compression artifacts, and jointly super-resolving low-resolution videos. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art methods on various video frame interpolation and enhancement tasks.

In recent years, the detection of distinctive objects in stereoscopic 3D images has drawn increasing attention. Unlike 2D salient object detection, salient object detection in stereoscopic 3D images is highly challenging. Hence, we propose a novel Deep Convolutional Residual Autoencoder (DCRA) for end-to-end salient object detection in stereoscopic 3D images. The core trainable architecture of the salient object detection model employs raw stereoscopic 3D images as the inputs and their corresponding ground truth saliency masks as the labels. A convolutional residual module is applied to both the encoder and the decoder as a basic building block in the DCRA, and long-range skip connections are employed to bypass the equal-sized feature maps between the encoder and the decoder. To explore the complex relationships and exploit the complementarity between RGB (photometric) and depth (geometric) information, multiple feature map fusion modules are constructed. These modules integrate texture and structure information between the RGB and depth branches of the encoder and fuse their features over several multiscale layers. Finally, to efficiently optimize DCRA parameters, a supervision pyramid based on boundary loss and background prior loss is adopted, which employs supervised learning over the multiscale layers in the decoder to prevent vanishing gradients and accelerate the training at the fusion stage. We compare the proposed DCRA with state-of-the-art methods on two challenging benchmark datasets. The results of these experiments demonstrate that our proposed DCRA performs favorably against the comparison models.

Multi-view based 3D reconstruction aims to obtain 3D structure information of objects in space through two-dimensional images. In this paper, we propose a new multi-view stereo network that can robustly reconstruct the scene. To enhance the feature representation ability of Point-MVSNet, a pyramid attention module is introduced. Specifically, we exploit the attention mechanism for the multi-scale feature pyramid to capture larger receptive fields and richer information. Instead of constructing a feature pyramid as the input, results of the pyramid attention module at different scales are directly used for the next layer. The network eventually generates a high-quality depth estimation for 3D reconstruction from sparse to dense by an iterative refinement schedule. Experiments have been performed to evaluate 3D reconstruction quality by comparison with existing state-of-the-art methods on the DTU dataset. The experimental results indicate our method performs the best in overall quality compared with previous methods, proving the effectiveness of our method. In the end, we use the data collected by mobile devices to implement 3D reconstruction with a combination of traditional and learning-based methods, providing ideas for the 3D reconstruction technology on mobile devices.

Traditional deep learning detection network has poor effect on the detection of infrared dim and small targets on the sea in the case of interference or bad weather. In this paper, an improved dim and small infrared ship detection network based on Haar wavelet is proposed. The HaarConv module is designed based on the high-frequency features obtained by Haar wavelet decomposition, which further increases the feature extraction ability of the backbone network for small targets. Meanwhile, the HaarUp-HaarDown module is designed by using Haar forward and inverse transform, replacing the up-sampling layer of the feature pyramid network and the down-sampling layer of backbone network to retain smaller target features. Furthermore, the pseudo-label-based method enables the network to conduct semi-supervised learning, which reduces the labeling cost and improves detection accuracy while expanding the amount of training data. The above method is applied to the YOLOv5-s lightweight network and 11278 infrared images (3352 labeled) of dim and small ships are collected as a dataset. The results show that the introduction of semi-supervised training method effectively expands the training dataset, and the mAP@.5:.95 increases by 23.5%. The proposed Haar wavelet improvement method can effectively improve the detection accuracy of dim and small infrared ship targets by more than 2%, and the number of parameters increases by only about 0.02M. Compared with existing methods, the proposed method reaches the state-of-the-art result and has good generalization performance.

This paper introduces a novel bi-directional con-volutional framework to cope with the large-variance scale problem in scene text detection. Due to the lack of scale normalization in recent CNN-based methods, text instances with large-variance scale are activated inconsistently in feature maps, which makes it hard for CNN-based methods to accurately locate multi-size text instances. Thus, we propose the relationship network (R-Net) that maps multi-scale convolutional features to a scale-invariant space to obtain consistent activation of multi-size text instances. Firstly, we implement an FPN-like backbone with a Spatial Relationship Module (SPM) to extract multi-scale features with powerful spatial semantics. Then, a Scale Relationship Module (SRM) constructed on feature pyramid propagates contextual scale information in sequential features through a bi-directional convolutional operation. SRM supplements the multi-scale information in different feature maps to obtain consistent activation of multi-size text instances. Compared with previous approaches, R-Net effectively handles the large-variance scale problem without complicated post processing and complex hand-crafted hyperparameter setting. Extensive experiments conducted on several benchmarks verify that our R-Net obtains state-of-the-art performance on both accuracy and efficiency. More specifically, R-Net achieves an F-measure of 85.6% at 21.4 frames/s and an F-measure of 81.7% at 11.8 frames/s for ICDAR 2015 and MSRA-TD500 datasets respectively, which is the latest SOTA. The code is available on https://github.com/wangyuxin87/R-Net .

Surface defect segmentation of no-service rail is important for its quality assessment. There are several challenges of uneven illumination, complex background, and difficulty of sample collection for no-service rail surface defects (NRSDs). In this article, we propose an acquisition scheme with two lamp light and color scan line charge-coupled device (CCD) to alleviate uneven illumination. Then, a multiple context information segmentation network is proposed to improve NRSD segmentation. The network makes full use of context information based on dense block, pyramid pooling module, and multi-information integration. Besides, the attention mechanism is applied to optimize extracted information by filtering noise. For the problem of real sample shortage, we propose to utilize artificial samples to train the network. And an NRSD data set NRSD-MN is built with artificial NRSDs and natural NRSDs. Experimental results show that our method is feasible and has a good segmentation effect on artificial and natural NRSDs.

Enhancement of low-light images is a challenging task due to the impact of low brightness, low contrast, and high noise. The inability to collect natural labeled data intensifies this problem further. Many researchers have attempted to solve this problem using learning-based approaches; however, most models ignore the impact of noise in low-lit images. In this paper, an encoder-decoder architecture, made up of separable convolution layers that solve the issues encountered in low-light image enhancement, is proposed. The architecture is trained end-to-end on a custom low-light image dataset (LID), comprising both clean and noisy images. We introduce a unique multi-context feature extraction module (MC-FEM) where the input first passes through a feature pyramid of dilated separable convolutions for hierarchical-context feature extraction followed by separable convolutions for feature compression. The model is optimized using a novel three-part loss function that focuses on high-level contextual features, structural similarity, and patch-wise local information. We conducted several ablation studies to determine the optimal model for low-light image enhancement under noisy and noiseless conditions. We have used performance metrics like peak-signal-to-noise ratio, structural similarity index matrix, visual information fidelity, and average brightness to demonstrate the superiority of the proposed work against the state-of-the-art algorithms. Qualitative results presented in this paper prove the strength and suitability of our model for real-time applications.

The segmentation effect of welding zone is vulnerable to image quality and target shape, which brings difficulty for the radiographic inspection of water-cooled pipe. A boundary-aware semantic segmentation algorithm is proposed to improve the precision and robustness of the welding zone inspection system. First, a convolutional neural network and atrous spatial pyramid pooling are employed to generate the feature map of the input image. Second, a boundary distance field (BDF) regression module is designed to predict the distance fields of the inner and outer boundaries, which can reflect the continuous boundary information of welding zone. Then, spatial attention weight is calculated according to the regressed boundary distance fields, and the calculated weight is fused into the feature map to enhance the significance of welding zone. Finally, the segmentation mask is predicted based on the fused feature map. Experiments indicate that the guidance of boundary information in the semantic segmentation algorithm can effectively reduce the interference of nonweld zones. The segmentation accuracy of the welding zone is 93.70%, and average detection time of a single image is less than 11 ms.

Reidentifying an occluded person across nonoverlapping cameras is still a challenging task. In this work, we propose a novel pose-guided part-based adaptive pyramid neural network for occluded person reidentification. Firstly, to alleviate the impact of occlusion, we utilize pose landmarks to generate pose-guided attention maps. The attention maps will help the model focus on the nonoccluded regions. Secondly, we use pyramid pooling to extract multiscale features in order to address the scale variation problem. The generated pyramid features are then multiplied by attention maps to achieve pose-guided adaptive pyramid features. Thirdly, we propose a pose-guided body part partition scheme to deal with the alignment problem. Accordingly, the adaptive pyramid features are divided into partitions and fed into individual fully connected layers. In the end, all the part-based matching scores are fused with a weighted sum rule for person reidentification. The effectiveness of our method is clearly validated by the experimental results on two popular occluded and holistic datasets, i.e., Occluded-DukeMTMC and the Market-1501.

Object detection is one of the core tasks in computer vision. Object detection algorithms often have difficulty detecting objects with diverse scales, especially those with smaller scales. To cope with this issue, Lin et al. proposed feature pyramid networks (FPNs), which aim for a feature pyramid with higher semantic content at every scale level. The FPN consists of a bottom-up pyramid and a top-down pyramid. The bottom-up pyramid is induced by a convolutional neural network as its layers of feature maps. The top-down pyramid is formed by progressive up-sampling of a highly semantic yet low-resolution feature map at the top of the bottom-up pyramid. At each up-sampling step, feature maps of the bottom-up pyramid are fused with the top-down pyramid to produce highly semantic yet high-resolution feature maps in the top-down pyramid. Despite significant improvement, the FPN still misses small-scale objects. To further improve the detection of small-scale objects, this paper proposes scale adaptive feature pyramid networks (SAFPNs). The SAFPN employs weights chosen adaptively to each input image in fusing feature maps of the bottom-up pyramid and top-down pyramid. Scale adaptive weights are computed by using a scale attention module built into the feature map fusion computation. The scale attention module is trained end-to-end to adapt to the scale of objects contained in images of the training dataset. Experimental evaluation, using both the 2-stage detector faster R-CNN and 1-stage detector RetinaNet, demonstrated the proposed approach’s effectiveness.

Surface defect detection is a critical task in industrial production process. Nowadays, there are lots of detection methods based on computer vision and have been successfully applied in industry, they also achieved good results. However, achieving full automation of surface defect detection remains a challenge, due to the complexity of surface defect, in intraclass. While the defects between interclass contain similar parts, there are large differences in appearance of the defects. To address these issues, this article proposes a pyramid feature fusion and global context attention network for pixel-wise detection of surface defect, called PGA-Net. In the framework, the multiscale features are extracted at first from backbone network. Then the pyramid feature fusion module is used to fuse these features into five resolutions through some efficient dense skip connections. Finally, the global context attention module is applied to the fusion feature maps of adjacent resolution, which allows effective information propagate from low-resolution fusion feature maps to high-resolution fusion ones. In addition, the boundary refinement block is added to the framework to refine the boundary of defect and improve the result of the prediction. The final prediction is the fusion of the five resolutions fusion feature maps. The results of evaluation on four real-world defect datasets demonstrate that the proposed method outperforms the state-of-the-art methods on mean intersection of union and mean pixel accuracy (NEU-Seg: 82.15%, DAGM 2007: 74.78%, MT_defect: 71.31%, Road_defect: 79.54%).

Visual surface defect inspection for metal part has become a rapidly developing research field within the last decade. But due to the variances of defect shapes and scales, the inspection of tiny and irregular shape defects has posed challenges on the robustness of the inspection model. In this context, a deep learning method based on the deformable convolution and concatenate feature pyramid (CFP) neural networks is proposed to improve the inspection. We design a deformable convolution layer in the neural networks as an attention mechanism to adaptively extract the features of defect shape and location, which enhances the inspection of the defects with large shape variances. We also merge the multiple hierarchical features collected from different deformable convolution layers by the CFP, which improves the inspection of tiny defects. The results show that the proposed method has a better generalization ability than traditional convolution neural networks.

Object detectors have improved in recent years, obtaining better results and faster inference time. However, small object detection is still a problem that has not yet a definitive solution. The autonomous weapons detection on Closed-circuit television (CCTV) has been studied recently, being extremely useful in the field of security, counter-terrorism, and risk mitigation. This article presents a new dataset obtained from a real CCTV installed in a university and the generation of synthetic images, to which Faster R-CNN was applied using Feature Pyramid Network with ResNet-50 resulting in a weapon detection model able to be used in quasi real-time CCTV (90 ms of inference time with an NVIDIA GeForce GTX-1080Ti card) improving the state of the art on weapon detection in a two stages training. In this work, an exhaustive experimental study of the detector with these datasets was performed, showing the impact of synthetic datasets on the training of weapons detection systems, as well as the main limitations that these systems present nowadays. The generated synthetic dataset and the real CCTV dataset are available to the whole research community.

Accurate and reliable segmentation of the prostate, its inner and surrounding structures in magnetic resonance (MR) images, is of essential importance for image-guided prostate interventions and treatments. Further clinical demand is the automated segmentation of central gland (CG) and peripheral zone (PZ). Although T2-weighted (T2W) images can show prostate tissues more clearly, the lack of contrast between PZ and other surrounding tissues in T2W images increases the difficulty of PZ segmentation. In this context, it is necessary to consider using multi-parametric MR images to obtain complementary information for prostate zonal segmentation. In this paper, we propose a 3D multi-scale discriminative network with pyramid attention module (PAM) and residual refinement block (RRB) for automated and accurate segmentation of CG and PZ using bi-parametric (T2W and apparent diffusion coefficient) MR images. One of the major difficulties in prostate MR image segmentation is the ambiguous edge between the prostate and other surrounding anatomical structures, which is reflected in some specific directions. Therefore, we design a multi-directional edge loss to help the network focus on the multi-directional edge information of foreground areas. For the Prostate Multi-parametric MRI (PROMM) dataset, our proposed model achieved Dice similarity coefficient of 0.908 at CG and 0.785 at PZ. The average boundary distance obtained by our model is 1.397 mm at CG and 3.891 mm at PZ. For the NCI-ISBI dataset, our method greatly improves the Dice similarity coefficient at PZ, reaching 0.806 and achieved the Dice similarity coefficient of 0.901 at CG. The experimental results on two different MR prostate datasets demonstrate that our model is more sensitive to object boundaries and outperforms other state-of-the-art methods. The visualization of feature map activations in PAM shows that the proposed model can capture multi-scale discriminative features effectively.

Recovering the 3D shape of the surgical site is crucial for multiple computer-assisted interventions. Stereo endoscopes can be used to compute 3D depth but computational stereo is a challenging, no...

Direct regression methods have demonstrated their success on various multi-oriented benchmarks for scene text detection due to the high recall rate for small targets and the direct regression for text boxes. However, too many false positive candidates and inaccurate position regression still limit the performance of these methods. In this paper, we propose an end-to-end method by introducing position-sensitive segmentation into the direct regression method to overcome these shortcomings. We generate the ground truth of position-sensitive segmentation maps based on the information of text boxes so that the position-sensitive segmentation module can be trained synchronously with the direct regression module. Besides, more information about the relative position of text is provided for the network through the training of position-sensitive segmentation maps, which improves the expressiveness of the network. We also introduce spatial pyramid of position-sensitive segmentation into the proposed method considering the huge differences in sizes and aspect ratios of scene texts and we propose position-sensitive COI(Corner area of Interest) pooling into the proposed method to speed up the inference. Experiments on datasets ICDAR2015, MLT-17 and COCO-Text demonstrate that the proposed method has a comparable performance with state-of-the-art methods while it is more efficient. We also provide abundant ablation experiments to demonstrate the effectiveness of these improvements in our proposed method.

Blur detection aims to detect the regions where the image is blur and identifies the blurred regions accurately. Since the traditional hand-crafted feature based methods usually are not robust enough to handle various complex scenarios, the study of blur detection problem remains to be a challenging task in terms of the accuracy and effectiveness of blur separation. In this paper, instead of seeking a new hand-crafted feature often seen in common approaches, we present a new deep feature-learning method by using feature pyramid network via recurrent Distinction Enhanced Block modules to solve the blur detection problem. We show that the newly introduced Distinction Enhanced Block is able to merge the high-level semantic information with the low-level details effectively, while at the same time, it is capable to keep a desirable identification between the blurry and clear regions. We also develop a new boundary penalty term to refine the boundaries of our results, which leads to more accurate detection and segmentation, compared with current available results in literature. Furthermore, due to the lacking of public datasets for blur detection problems, we have established a new blur detection dataset which may enrich the experimental benchmarks for testing. In terms of efficiency associated with the competitive performance, in particular, our model is shown to have best-in-class evaluation speed, about 0.14 second to evaluate an input image in regular personal laptop. The performance evaluations on both CUHK dataset and our SZU-BD dataset validate that the proposed model outperforms various existing state-of-the-arts methods through commonly used metric indexes.

Scale variation is one of the major challenges in object detection task. Modern region-based object detection architectures often adopt Feature Pyramid Network (FPN) as feature extraction neck to achieve multi-scale feature representation in solving scale variation problem. However, due to the rough feature selection strategy in Region of Interest (RoI) feature extraction step, these methods might not perform well on object detection under strong scale variation. In this work, we are motivated by the limitations of current FPN-based two-stage object detectors and then present a novel module, namely scale-aware feature selective (SAFS) module, that flexibly and adaptively selects feature levels in two-stage object detectors. Specifically, we firstly build the RoI Pyramid in standard FPN structure to extract RoI features from various scale levels. Next, in order to achieve scale-aware mechanism for solving scale variation issue, we develop a novel weighting gate function containing one set of trainable parameters to automatically learn the fusion weight for each RoI feature level, which relieves the limitation of hard feature selection strategy guided by online instance size. Outputs from the RoI features with the learned weights are fused for classification and bounding box regression. Furthermore, we design a multi-level SAFS architecture to obtain different types of RoI feature combinations that ensures our method is more robust to various instance scales. Experimental results show that our SAFS module is very compatible with most of two-stage object detectors and could achieve state-of-the-art results with Average Precision of 48.3 on COCO test-dev and other popular object detection benchmarks. Our code will be made publicly available.