Search

• We design a novel semantic-refined spatial pyramid network (SSPNet). • The SSPNet can benefit the fusing of multi-scale information. • The SSPNet can enhance the representation ability of deep semantic features. • The multi-scale feature will be adaptively integrated by our method. • The texture features are obtained to enrich the multi-scale features. [Display omitted] In this paper, we propose a novel encoder-decoder model called Semantic-refined Spatial Pyramid Network (SSPNet) for generating high-quality density maps, which aims to build a scale-aware counting network to estimate the number of crowds accurately. The SSPNet consists of the front-end based on VGG-16, spatial pyramid multi-scale module (SPMM), and semantic enhancement module (SEM). First, a series of convolutional neural layers are utilized as the front-end to get deeper features without the extra computational cost. Moreover, the SPMM, which has a spatial pyramid structure with multiple receptive fields, is employed to capture multi-scale features. Furthermore, the SEM is designed to refine the features captured by SPMM, which uses deep semantic information to better integrate multi-scale features. Finally, the shallow texture information is adopted to compensate for the detail of the feature map to enhance the quality of the density map. Extensive experiments and comparisons on three challenge datasets, including ShanghaiTech Part_A & Part_B, UCF_CC_50, and UCF-QNRF, illustrate the superiority of our method. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) has been a challenging task in the field of artificial intelligence. In this paper, we propose a novel model, named regional self-attention convolutional neural network (RSACNN), for FER. Different from the previous methods, RSACNN makes full use of the facial texture of expression salient region, so yields a robust feature representation for FER. The proposed model contains two novel parts: regional local multiple pattern (RLMP) based on the improved K-means algorithm and the regional self-attention module (RSAM). First, RLMP uses the improved K-means algorithm to dynamically cluster the pixels to ensure the robustness of texture features with expression salient variation. Besides, the texture description is enhanced by extending the binary pattern to the multiple patterns and integrating the information of gray difference between pixels in the region. Next, RSAM can adaptively form weights for each region through the self-attention mechanism, and use rank regularization loss (RRLoss) to constrain the weights of different regions. By jointly combining RLMP and RSAM, RSACNN can effectively enhance the feature representation of expression salient regions, so that the performance of expression recognition can be improved. Extensive experiments on public datasets, i.e. CK + , Oulu-CASIA, Fer2013 and SFEW, prove the superiority of our method over state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Crowd counting is essential for video surveillance and public safety. The performance of counting models has been greatly improved with the rapid development of Convolution neural networks (CNN), while it still suffers interference from complex background and large-scale variation. To relieve the above challenges, this paper proposes a novel Multi-branch Progressive Embedding Network (MPENet) for crowd counting. Specifically, the proposed network mainly consists of two modules named Background Area Filter (BAF) and Sequential Multi-scale Modules (SMM), which are embedded with each other to generate higher-quality density maps. Firstly, the BAF model base on attention mechanism is proposed to distinguish crowd from background, which effectively avoids the model outputting positive predictions in the background region. Meanwhile, a multi-level supervision mechanism is proposed to generate more accurate attention maps. Besides, the SMM module is designed to be progressively embedded with scale context information so that the scale feature will be smooth and continuous. Finally, a novel multi-scale consistency structural loss is proposed to avoid pixel-level isolation due to Euclidean loss. The proposed method significantly improves counting accuracy, achieving Mean Absolute Error (MAE) of 57.6 and 6.9 on ShanghaiTechA and ShanghaiTechB respectively. • The insufficient scale hierarchy imposes some limitations on the performance. • The network is inclined to confuse dense regions and background regions. • Euclidean distance is strict pixel-level alignment will overfit the density map noise. • Utilize attention mechanisms to filter background noise [ABSTRACT FROM AUTHOR]

Recently, region proposal network (RPN) has been combined with the Siamese network for tracking and shown excellent accuracy and high efficiency. However, the low correlation between the classification score and localization accuracy in tracking has weakened the performance of the tracking model. In this paper, we propose an IoU-guided Siamese RPN (SiamIG) to address this problem. Specifically, SiamIG predicts the intersection-over-union (IoU) with the ground truth for each regressed anchor by using an IoU predictor. Then the predicted IoU is multiplied with the classification score to compute the final score, which is more suitable for localization accuracy. Furthermore, each anchor in the regression branch is assigned an IoU-based weighting factor such that the tracking accuracy can be further improved. Specifically, anchor boxes with high IoU are given more attention because of the IoU-based weighting factor, which helps the model pay more attention to an anchor box with a high IoU. Experiments demonstrate that the proposed method runs at over 200 fps and achieves up to 2.4%(AUC), 3%(EAO) and 2.3%(AUC) improvements over the baseline on the OTB-100, VOT-2016 and UAV123 datasets respectively. [ABSTRACT FROM AUTHOR]

One primary challenge in liver segmentation is the fuzzy edge contour. Recently, fully convolutional neural networks (FCNs) have been widely used in liver segmentation because of their adequate feature extraction. Nevertheless, the context among liver slices is still ignored by FCN. To address this issue, we first propose a bidirectional convolutional long short-term memory (BiConvLSTM) to explore contextual information. Meanwhile, the attention gate (AG) is utilized to fuse high-dimensional information from BiConvLSTM to remove irrelevant features. Besides, Shape-Net network is proposed to extend the liver shape pattern by latent space information, which will contribute to reduce the interference of fuzzy boundaries. Finally, the improved active contour loss function with L2 norm acts as a feature constraint. Experimental results on public benchmark datasets show that the proposed method slightly outperforms other newly published methods and achieves good performance for liver segmentation. [ABSTRACT FROM AUTHOR]

• We propose a novel multi-task collaboration network (MTCNet) to deal with faces with large rotation faces by cooperation between face detection and face alignment, which avoids the semantic ambiguity among RIP angle groups well. • We incorporate the angle information to optimize the loss function of face alignment to improve the efficiency of model convergence. • We introduce the contextual information to enhance the quality of feature maps for mining those faces with serious rotation change from unconstrained scenarios. • Extensive experiments are conducted to demonstrate the effectiveness of the proposed MTCNet on FDDB and WIDER FACE. Detecting rotated faces is a challenging task with images from uncontrolled environments. The use of deep convolutional neural networks have greatly improved detection performance, but these methods still do not fully exploit face structure information. This leaves faces with more extreme rotation angles undetectable. In this paper, we present a novel Multi-Task Collaboration Network (MTCNet) for rotation-invariance face detection that fully uses facial landmarks to improve the detection performance by means of collaboration between face detection and face alignment. Differing from previous methods that predict rotation angles in a single step, MTCNet employs a cascaded architecture with three stages to predict faces with gradually decreasing rotation-in-plane ranges in a coarse-to-fine process. Accurate facial landmarks further facilitate face detection. We also introduce a new training loss by integrating the geometric angle into the penalization process, which is much more reasonable than measuring the differences of training samples roughly. Our approach also explores contextual information to distinguish challenging faces from unconstrained scenarios. Extensive experimental results were conducted to demonstrate the effectiveness of MTCNet on both the multiple orientation and rotation datasets. Empirical studies show that MTCNet achieves results competitive with state-of-the-art face detectors while being time-efficient. [ABSTRACT FROM AUTHOR]

The image classification precision is vastly enhanced with the growing complexity of convolutional neural network (CNN) structures. However, the uneven visual separability between categories leads to various difficulties in classification. The hierarchical structure of categories can be leveraged to deal with it, but a few CNNs pay attention to the character of data. Besides, a network model with a hierarchical structure is promising to extract more specific features from the data than current CNNs, since, for the latter, all categories have the same fixed number of layers for feed-forward computation. In this paper, we propose to use category hierarchies to integrate ResNet-style modules to form a hierarchical network model in a top-down manner. To extract abundant discriminative features and improve the computation efficiency, we adopt residual block selection based on coarse categories to allocate different computation paths. Each residual block works as a switch to determine the JUMP or JOIN mode for an individual coarse category. Interestingly, since some categories need less feed-forward computation than others by jumping layers, the average inference time cost is reduced. Extensive experiments show that our hierarchical network achieves higher prediction accuracy with similar FLOPs on CIFAR-10 and CIFAR-100, SVHM, and Tiny-ImageNet datasets compared to original residual networks and other existing selection inference methods. • The hierarchy of classes is used to form a hierarchical network. • The selective computation based on coarse category is proposed for the first time. • The proposed network model helps inference acceleration and accuracy improvement. [ABSTRACT FROM AUTHOR]