SMCA-CNN: Learning a Semantic Mask and Cross-Scale Adaptive Feature for Robust Crowd Counting

Density-based crowd counting methods with deep convolutional neural network (CNN) have achieved the state of the art on the challenging datasets. Experimental results showed that the performance of these methods suffers from two problems: 1) Background interference problem: there are some estimated spurious density values in the background regions which degrade the counting accuracy. 2) Cross-scale problem: the scale of human heads varies greatly in crowd images which lead to poorer quality of the density maps. In this study, we aim to address the two problems for enhancing the counting accuracy. To address the former, a light semantic mask module (SMM) is proposed to learn the semantic masks of crowd images where the ground-truth semantic masks generated from the ground-truth density map are taken as the supervision information. To tackle the latter, we propose a span architecture (SA) to effectively capture the large-scale-variation information in the crowd images by building the cross-scale features from the pyramidal structure of a deep CNN. To adaptively leverage the salient cross-scale features, a Cross-scale Adaptive Module (CAM) is delicately designed. In the end, integrating all elements above, an end-to-end trainable and single-column crowd counting model called the SMCA-CNN is developed and trained with a joint loss function consisting of the cross-entropy loss and Euclidean loss. Extensive experiments on five challenging datasets demonstrate the effectiveness of our SMCA-CNN. Compared with the previous state-of-the-art methods, our model achieves 17.1% lower MAE on dataset UCF_CC_50 and 23.6% lower MAE on the newly published dataset UCF-QNRF.