Search

Recently, the research of crowd counting has attracted increasing attention but still faces many challenges, such as crowded scenes, scale variations and cluttered backgrounds. With the development of deep learning, density maps are widely used for crowd counting, where the quality of density maps plays a crucial role in counting performance. In this paper, we propose a new convolutional network architecture, called double multi-scale feature fusion network (DMFFNet), to generate high-quality density maps and accurate counting estimates. DMFFNet utilizes VGG19 to extract multi-scale feature maps from input images. The features from last three scales are further enlarged the receptive fields by three designed dilated feature pyramid modules, and then fused together. Moreover, a feature enhancement module composed of spatial attention and channel-wise attention is presented to weight the fused feature maps for effectively distinguishing between crowd and background. We also design a new dual-scale loss to optimize the network during training. Experimental results show that DMFFNet reduces MAEs by at least 1.5 % , 1.5 % , 1.2 % , 0.6 % and 0.5 % on UCF _ CC _ 50, UCF-QNRF, JHU-Crowd++, ShanghaiTech Part A and Part B datasets, and decreases MSEs by at least 1.8 % and 0.1 % on JHU-Crowd++ and ShanghaiTech Part B datasets, as compared with the state-of-the-art. [ABSTRACT FROM AUTHOR]

As a fundamental computer vision task, crowd counting plays an important role in public safety. Currently, deep learning based head detection is a promising method for crowd counting. However, the highly concerned object detection networks cannot be well applied to this problem for three reasons: (1) Existing loss functions fail to address sample imbalance in highly dense and complex scenes; (2) Canonical object detectors lack spatial coherence in loss calculation, disregarding the relationship between object location and background region; (3) Most of the head detection datasets are only annotated with the center points, i.e. without bounding boxes. To overcome these issues, we propose a novel Mask Focal Loss (MFL) based on heatmap via the Gaussian kernel. MFL provides a unifying framework for the loss functions based on both heatmap and binary feature map ground truths. Additionally, we introduce GTA_Head, a synthetic dataset with comprehensive annotations, for evaluation and comparison. Extensive experimental results demonstrate the superior performance of our MFL across various detectors and datasets, and it can reduce MAE and RMSE by up to 47.03% and 61.99%, respectively. Therefore, our work presents a strong foundation for advancing crowd counting methods based on density estimation. [ABSTRACT FROM AUTHOR]

Crowd counting from a single image is a challenging task due to perspective distortion and large-scale variation in crowd scenes. Many Researches only focus on local features to create density maps which is not effective in handing the challenges. This paper proposes a novel network named global-to-local adaptive spatial encoder network, which focuses on global features to generate a total structure density map of the population distribution, and then utilizes local features to reconstruct the total structure density map in detail to generate high-quality density map. To capture global features, local information and correlate them, we design a contextual module using different kernels with convolution and transposed convolution. To create a density map from global structure to local detail, two branches are designed, the global distribution branch and the local detail branch. The former aims to capture the population distribution region of interest in terms of global structure, and the latter aims to focus on the local details of each unit. Furthermore, to overcome the problem of pixel-wise loss of MSE, this paper proposes an efficient loss function that focuses on perceiving the possible crowd distribution over the whole image. We also apply a new upsampling mechanism that learns to create high-quality density maps on its own is advisable. The proposed network can capture the characteristics of pedestrian distribution and predict accurate results. It is evaluated on four crowd counting datasets (ShanghaiTech, NWPU, UCF_QNRF, UCF_CC_50), it obtains MAE of 67.1 and MSE, and achieves 108.8 in ShanghaiTech and gets MAE of 139.2 and the best MSE of 217.7 in UCF_CC_50 dataset and so on, and our method shows state-of-the-art on all the datasets. [ABSTRACT FROM AUTHOR]

Crowd counting is a practical yet essential research topic in computer vision, which has been beneficial to diverse applications in smart city environment safety. The commonly adopted paradigm in most existing methods is to regress a Gaussian density map that works as the learning objective during model training. However, given the unavoidable identity occlusion and scale variation in a crowd image, the corresponding Gaussian density map is degraded, failing to provide reliable supervision for optimization. To address this problem, we propose to replace the traditional Gaussian density map with a better alternation, namely the smooth inverse map (SIM). The proposed SIM can reflect the head location spatially and provide a smooth gradient to stabilize the model learning. Besides, we want the method to learn more discriminative features to cope with the challenge of large-scale variations. We deliver a multiscale aggregation (MA) to adaptively fuse features in different hierarchies to benefit semantic information under diverse receptive filed. The SIM and MA are meant to be complementary modules to guide the model in learning an accurate density map. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method compared with the state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

Crowd counting methods have become increasingly mature. However, the problem of dramatic scale variation still exists. For this reason, we propose an efficient separated and fused pyramid attention network, which can extract multiscale features on channels and space and greatly alleviate the problem of dramatic scale variation. First, in order to extract the rich features on the channel, we design a separated and fused channel attention module, which is composed of two 3x3 convolution layers, a separated attention module, and a SE module. Second, we design a spatial contextual feature fusion module to fully extract multiscale features in spatial dimensions. Finally, we conduct comparison experiments with state-of-the-art methods on several challenging datasets, including the ShanghaiTech, UCF_CC_50, and WorldExpo'10 datasets. The experimental results show our method outperforms most of the state-of-the-art methods. We conduct ablation experiments on the ShanghaiTech Part A and Part B datasets to verify the importance of each submodule. [ABSTRACT FROM AUTHOR]

As a challenging issue in computer vision, crowd counting has been increasingly studied. A convolutional neural network (CNN) is an effective system for handling crowd counting, based on constructing a CNN to generate a high-quality density estimation map. However, conventional CNN-based methods only consider the mapping from the crowd image to the density map, neglecting reconstruction from the density map to the crowd image and the impact of this reconstruction on the CNN performance. Here, we present a novel model denoted a dual-CNN (DualCNN) to improve the conventional CNN performance on crowd counting. Our DualCNN comprises a primal network for generating the density maps from the crowd image and a secondary network for reconstructing the crowd image from the density map. The two networks are trained through an iterative and alternating learning process, and the performance of the final model is improved by considering the interactions of the two networks. In addition, we introduce the attention mechanism into the dual network to enhance the primal network robustness against the background influence of the crowd image. The experimental results indicate that the proposed method significantly improves the performance of CNNs in crowd counting. [ABSTRACT FROM AUTHOR]

Traditional approach of mobile crowd estimation involves counting a group of individuals at a specific place, manually, in real-time. It is a laborious exercise that can be physically and mentally demanding. In Hong Kong, a large rally can last more than six hours, making the manual count method susceptible to human errors. While crowd counting using object detection and tracking has been well-established in computer vision, such application has remained relatively small scale within a controlled indoor setting (e.g. counting people at fixed gateways in a mall). No attempt to date has applied the automatic crowd counting method to count hundreds of thousands of people along an open stretch of rally route within the complex urban outdoor landscape. This research proposed an integrated approach that combines the capture-recapture method in statistics and a Convolutional Neural Network (CNN) method in computer vision to count the mobile crowd. The research teams implemented the integrative approach and counted 276,970 people with a 95% confidence interval of 263,663 to 290,276 in the 2019, July 1st Rally in Hong Kong. This work counted the attendance of a large-scale rally as a proof of concept to fill in a gap in the empirical studies. The intellectual merits and research findings shed useful insights to improve mobile population estimation and leverage alternative data sources to support related scientific applications. [ABSTRACT FROM AUTHOR]

The number of people and the estimation of the population density are one of the important information concerned by intelligent monitoring. This article reviews, summarizes, and classifies the research methods and status of population statistics and population density estimation. According to the different methods of obtaining crowd information, the research methods of population density statistics and population density estimation are divided into detection-based methods, regression-based methods, and density-based methods. The methods based on density estimation are studied in detail. The network structures based on density estimation are divided into a single-column deep convolutional neural network and a multi-column convolutional neural network. Summarized the basic ideas, advantages and disadvantages of each method, analyzed and introduced the representative algorithms of each method, analyzed and compared related experiments, and introduced common mainstream data sets and performance evaluations for population statistics and population density estimation. Indicators and evaluation methods, and prospects for future possible research directions and corresponding development trend in this field. [ABSTRACT FROM AUTHOR]

The crowd counting problem aims to predict the number of pedestrians in a surveillance video or an image and produce a crowd density map. Achieving accurate crowd counting in different crowded scenes is still challenging due to drastic scale changes and severe occlusions. This paper proposes an efficient multi-scale contextual feature fusion network for counting crowds with varying densities and scales, abbreviated as MSC-FFN. We design a spatial pyramid feature extraction module that enables rich contextual feature extraction to adapt to rapid scale changes. To further enhance the model's ability to suppress background and focus on main features, we also design a spatial channel attention module to integrate feature map correlations from spatial and channel dimensions, so that the network can focus on main crowd features and filter out irrelevant background information, which outputs a high-quality density map. We conduct extensive experiments on multiple challenging crowds counting datasets, including UCF_CC_50, ShanghaiTech, WorldExpo'10, and Mall dataset, and the results demonstrate that MSC-FFN outperforms many state-of-the-art methods in counting performance and generated density maps. [ABSTRACT FROM AUTHOR]

Recently, state-of-the-art crowd counting methods have focused more on predicting a density map and then obtaining the final aggregated count. In 2018, a typical density map-based network for congested scene recognition called CSRNet was proposed, and it achieved better crowd counting performance than previous methods with a simple architecture. It utilizes the first 10 layers from VGG-16 as the front end and deploys dilated convolutional layers as the back-end to generate high-quality density maps. CSRNet has been demonstrated on four datasets (ShanghaiTech dataset, the UCF_CC_50 dataset, the World Expo'10 dataset, and the UCSD dataset) and delivered great performance. To obtain better performance, in this paper, we propose a small network as a new component that generates a counting residual estimation, and we combine our component with CSRNet. We demonstrate this combined network on three datasets (ShanghaiTech dataset, the UCF_CC_50 dataset, and the World Expo'10 dataset) and compare the results with those of CSRNet. The results show that our method has significantly improved the results of CSRNet. Through a series of experiments, such as ablation experiments and control experiments, we demonstrate the effectiveness of our method. In the future, we will apply our method to other networks to achieve better results. [ABSTRACT FROM AUTHOR]

Crowd counting has long been a challenging task due to the perspective distortion and variability in head size. The previous methods ignore the multi-scale information in images or simply use convolutions with different kernel sizes to extract multi-scale features, resulting in incomplete multi-scale features extracted. In this paper, we propose a crowd counting model called Multi-scale Dilated Convolution of Feature Fusion Network (MsDFNet) based on a CNN (convolutional neural network). Our MsDFNet is based on the regression method of the density map. The density map is predicted by the parameters learned by CNN to obtain better prediction results. The proposed network mainly includes three components, a CNN to extract low-level features, a multi-scale dilated convolution module and multi-column feature fusion blocks, a density map regression module. Multi-scale dilated convolutions are employed to extract multi-scale high-level features, and the features extracted from different columns are fused. The combination of the multi-scale dilated convolution module and the multi-column feature fusion block can effectively extract more complete multi-scale features and boost the performance of counting small-sized targets. Experiments show that the problem of various head sizes in images can be effectively solved by fusing multi-scale context feature information. We prove the effectiveness of our method on two public datasets (The ShanghaiTech dataset and the UCF_CC_50 dataset). We compare our method with the previous state-of-the-art crowd counting algorithms in terms of MAE (Mean Absolute Error) and MSE (Mean Square Error) and significantly improves the performance, especially in case of various head sizes. On the UCF_CC_50 dataset, our method reduces the MAE index by 28.6 compared with the previous state-of-the-art method. (The lower the MAE, the better the performance). [ABSTRACT FROM AUTHOR]

Accurately modeling the crowd's head scale variations is an effective way to improve the counting accuracy of the crowd counting methods. Most counting networks apply a multi-branch network structure to obtain different scales of head features. Although they have achieved promising results, they do not perform very well on the extreme scale variation scene due to the limited scale representability. Meanwhile, these methods are prone to recognize background objects as foreground crowds in complex scenes due to the limited context and high-level semantic information. We propose a compositional multi-scale feature enhanced learning approach (COMAL) for crowd counting to handle the above limitations. COMAL enhances the multi-scale feature representations from three aspects: (1) The semantic enhanced module (SEM) is developed for embedding the high-level semantic information to the multi-scale features; (2) The diversity enhanced module (DEM) is proposed to enrich the variety of crowd features' different scales; (3) The context enhanced module (CEM) is designed for strengthening the multi-scale features with more context information. Based on the proposed COMAL, we develop a crowd counting network under the encoder-decoder framework and perform extensive experiments on ShanghaiTech, UCF_CC_50, and UCF-QNRF datasets. Qualitative and quantitive results demonstrate the effectiveness of the proposed COMAL. [ABSTRACT FROM AUTHOR]

Crowd counting has captured wide attention in computer vision, which aims to accurately count the number of people in still images or video scenes. However, it's still a challenging task due to the scale variation and cluttered background in crowd scenes. In this paper, we propose a 2-stage Double Attention convolutional neural network for crowd counting, and call it 2-DA-CNN, which could deal with scale variation and cluttered background in crowd counting. The proposed 2-DA-CNN includes three parts. The first part is the front-end module which consists of a set of convolution operations, whose function is to extract abundant feature of crowd. The second part is the first double attention module, which contains trunk branch and mask branch. The former is mainly composed by multi-column CNN module, which is to deal with scale variation in crowd scenes. The latter can generate two masks, which aims to assign interesting regions reasonably in cluttered situation. The third part is the second double attention module, similar to the first double attention module, which can enhance the performance of multi-column CNN module further. In addition, we propose progressive training method to improve the drawback of using geometry-adaptive kernels to generate ground truth. The experimental results on three mainstream datasets (ShanghaiTech part B, ShanghaiTech part A and UCF_CC_50) suggest that the proposed 2-DA-CNN is competitive with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Dense crowd counting and modeling at different gatherings has ignited a new flame in the visual surveillance research community. There is a high possibility of mishappenings in the form of stampede, mob fighting at these gatherings and the administration is helpless in these scenarios. There is a requirement of analyzing the crowd to prevent these dangerous situations. The proposed work is a case study of Kumbh Mela which models the crowd counting in densely populated images. In the proposed work, the orthographic projection of the crowd is captured using a camera attached to a drone, to reduce the effect of occlusion and scaling which, otherwise, may get introduce during image acquisition process. The captured data is fed to a Convolutional Neural Network for training the model to count head of persons present in the frame. The results obtained from the trained model are validated using geometry and imaging techniques. The proposed model has achieved a mean-absolute-error of 94.3 and a mean-squared-error of 104.6 which seems to outperform the existing state-of-the-art models with respect to the reported performance parameters. The proposed model can be used as a viable solution in applications related to modeling the crowd behavior. [ABSTRACT FROM AUTHOR]

Crowd counting has played a substantial role in intelligent surveillance. This work presents a multi-scale multi-task convolutional neural network (MSMT-CNN) to estimate accurate density maps, thus can count the crowd through summing up all values in the estimated density maps. The ground truth density maps used for training are generated by a novel adaptive human-shaped kernel. In addition to resolving the scale problem with the multi-scale strategy, the multi-task learning strategy is added so as to make the estimated density maps more accurate. A weighted loss function is proposed to enhance the activations in dense regions and suppress the background noise. Experimental results on two benchmarking datasets reveal the strong ability of MSMT-CNN. Compared with existing crowd counting methods, the root mean squared error is decreased by 39.8 on the UCF_CC_50 dataset, and the mean absolute error is decreased by 2.3 on the World Expo'10 dataset. Furthermore, the evaluations in practical bus videos verify the practicability of our MSMT-CNN. [ABSTRACT FROM AUTHOR]

Growing numbers of crowd density estimation methods have been developed in scene monitoring, crowd safety and on-site management scheduling. We proposed a method for density estimation of a single static image based on convolutional neural network naming Multi-scale Dilated Convolution of Convolutional Neural Network (Multi-scale-CNN). The proposed method employed the method of density maps regression to learn the mapping relationship between single-image and density maps through convolutional neural network. The adopted network structure is composed of two major components to adapt changes of characters scales in crowd images, a convolutional neural network for the general feature extraction and the other is multi-scale dilated convolution for disposing the scale change problem. It is insufficient for currently study that tackled the multi-column or multi-input convolutional neural networks to solve multi-scale problems. Our method utilizes a single-column network to extract features and combines multi-scale dilated convolution to aggregate multi-scale information to address the shortcomings of two networks. The multi-scale dilated convolution module aggregates multi-scale context information systematically by making use of dilated convolution without reducing the receiving domain, thereby integrate the underlying detail information into the high-level semantic features to promote the perception and counting ability of network for small targets. This paper demonstrates the proposed network structure in ShanghaiTech dataset, UCF_CC_50 dataset and worldexpo'10 dataset, and compares the results with numbers of current mainstream crowd counting algorithms, proves that our method surpasses current state-of-the-art methods and has excellent counting accuracy and robustness. The training and testing codes of our method models can be downloaded at https://github.com/doctorwgd/Multi-scale-CNN. [ABSTRACT FROM AUTHOR]

Automatic counting for moving crowds in digital images is an important application in computer artificial intelligence, especially for safety and management purposes. This paper presents a new method to estimate the size of a crowd. The new algorithm depends on sequential frame differences to estimate the crowd size in a scene. However, relying only on these simple differences adds more constraints for extracting sufficient crowd descriptors. A curvelet transform is employed to achieve that goal. Every two sequential frames are transformed into multi-resolution and multi-direction formats, and then the frame differences are detected at every subband in the curvelet domain. Statistical features out of each subband are then calculated, and the collected features from all subbands are considered as a descriptor vector for the crowd in the scene. Finally, a neural network is manipulated to map the descriptor vectors into predicted counts. The experimental results show that the proposed curvelet statistical features are more robust and provide crowd counting with higher accuracy than previous approaches. [ABSTRACT FROM AUTHOR]

Reliable and real-time crowd counting is one of the most important tasks in intelligent visual surveillance systems. Most previous works only count passing people based on color information. Owing to the restrictions of color information influences themselves for multimedia processing, they will be affected inevitably by the unpredictable complex environments (e.g. illumination, occlusion, and shadow). To overcome this bottleneck, we propose a new algorithm by multimodal joint information processing for crowd counting. In our method, we use color and depth information together with a ordinary depth camera (e.g. Microsoft Kinect). Specifically, we first detect each head of the passing or still person in the surveillance region with adaptive modulation ability to varying scenes on depth information. Then, we track and count each detected head on color information. The characteristic advantage of our algorithm is that it is scene adaptive, which means the algorithm can be applied into all kinds of different scenes directly without additional conditions. Based on the proposed approach, we have built a practical system for robust and fast crowd counting facing complicated scenes. Extensive experimental results show the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Reliable people counting is a crucial task in video surveillances. Among the available techniques, map-based approaches have shown a good performance in estimating the number of people in crowds. These approaches generally subtract the background, and then map the number of people to some features such as foreground area, texture features or edge count. However, in complex scenes, they suffer from inaccurate foreground/background segmentations, erroneous image features, and require large amount of training data to capture the wide variations in crowd distribution. This paper proposes a method using motion statistics of feature-points to estimate the number of moving people in a crowd. Simple feature-points are tracked within the scene. Then moving feature-points are partitioned into clusters corresponding to separate groups of people. For each group, three statistical features are calculated from related feature-points. The amount of moving feature-points is used to provide a rough estimate of group size. Furthermore, motion trajectories of feature-points are utilized to extract two other features related with the amount of occlusions present in groups. The extracted data are used to estimate the number of people in each group, so that the total crowd size is the sum of all group estimates. The experimental results show that the proposed method outperforms the state of the art approaches, e.g., with MSE of 2.357 and MAE of 1.093 for the benchmark video clip 'Peds1'. The proposed approach is good for estimating the number of people in public places, such as pedestrian walkways and parks, where people are moving and partial occlusions present in the scene. [ABSTRACT FROM AUTHOR]