Search

• SeSAME system for collecting and analysing the behaviours of visitors inside a museum environment. • SeSAME is based on a multi-modal deep neural network architecture. • SeSAME uses re-identification techniques to perform visual profiling of visitor interest. • SeSAME aims at developing a tailored user experience in museum environment. Nowadays, understanding and analysing visitors activities and behaviours is becoming imperative for personalising and improving the user experience in a museum environment. Users' behaviour can provide important statistics, insights and objective information about their interactions, such as attraction, attention and action. These data represent a precious value for the museum curators, and they are one of the parameters that need to be assessed. These information are collected through manual approaches based on questionnaires or visual observations. This procedure is time consuming and can be affected by the subjective interpretation of the evaluator. From such premises, SeSAME (Senseable Self Adapting Museum Environment) a novel system for collecting and analysing the behaviours of visitors inside a museum environment is presented in this paper. SeSAME is based on a multi-modal deep neural network architecture able to extract anthropometric and appearance features from RGB-D videos acquired in crowded environments. Our approach has been tested on four different temporal modelling methods to aggregate a sequence of image-level features into clip-level features. This paper uses as a benchmark TVPR2, a public dataset of acquired videos with an RGB-D camera in a top-view configuration, in the presence of persistent and temporarily heavy occlusion. Moreover, a dataset specifically collected for this work has been acquired in a real museum environment, which is Palazzo Buonaccorsi, an important historical building in Macerata, in Marche Region in the center of Italy. During the experimental phase, the evaluation metrics show the effectiveness and the suitability of the proposed method. [ABSTRACT FROM AUTHOR]

The paper focuses on developing the system for people flow recognition based on many video camera images. The project arose based on the real need — its application in the city malls to find the most visited places and support customers' campaigns. The potential application in other domains is possible (public buildings, airports). In contrast to the existing solutions, our approach involves on-edge image analysis to decrease the risk of data loss and the system cost. In the project, we design the whole processing pipeline composed of the component modules responsible for detecting, tracking, and reidentifying (shuffling) people. Our experimental platform enabled us to compare multiple method variants for each module. Based on extensive experimental research, the final solution uses the pretrained SSD method in the detection module. The centroid algorithm applied to displacement vectors combined with the Siamese network is the basis for the object tracking module. The best model to solve the reidentification task is the Resnet50. For the Market 1501 dataset, it achieved Rank-1 efficiency of 84.6%. The system gives a visualization of the main paths of people's movements in the form of a heat map and assigns the direction where people most often look. In the experimental study, we assessed the system's effectiveness and time efficiency, and the current results give a perspective for its commercialization in the nearest future. • Development of the whole processing pipeline of human flow and reidentification. • Research aiming at the choice of competitive component methods • Experimental tuning of methods to achieve the high efficiency and low processing time. • Extension of the centroids algorithm with displacement vectors. • Platform for easy testing various methods of detection and reidentification. [ABSTRACT FROM AUTHOR]