Search

Video stabilization is crucial for video representation learning, which suffers from the challenges such as the perception of unstable vision, the stripping and cognition of target motion features in complex scenes, the correction of the jittery camera systems trails. In this paper, we propose a Self-supervised sparse Optical Flow Transformer (SOFT) model, consisting of a self-supervised contrastive learning transformer network, a sparse optical flow perception network and a multimodal cognitive fusion network. The SOFT model takes advantage of optical flow to estimate motion. The sparse optical flow perception network perceiving partially sparse optical flow containing motion features. This serves as the input to the self-supervised contrastive learning transformer network for generating sparse optical flow features, which are fed into the multimodal cognitive fusion network together with the real and virtual camera pose for video frame warping. Experimental comparisons with state-of-the-art models on 4 metrics demonstrate the effectiveness of the SOFT model. It achieves the best performance with an average Stability of 0.869 and average Distortion of 0.993 across 6 categories videos, which shows that the SOFT model can effectively perceive the motion in the video and smooth the jitter track of videos. [ABSTRACT FROM AUTHOR]

The paper focuses on developing the system for people flow recognition based on many video camera images. The project arose based on the real need — its application in the city malls to find the most visited places and support customers' campaigns. The potential application in other domains is possible (public buildings, airports). In contrast to the existing solutions, our approach involves on-edge image analysis to decrease the risk of data loss and the system cost. In the project, we design the whole processing pipeline composed of the component modules responsible for detecting, tracking, and reidentifying (shuffling) people. Our experimental platform enabled us to compare multiple method variants for each module. Based on extensive experimental research, the final solution uses the pretrained SSD method in the detection module. The centroid algorithm applied to displacement vectors combined with the Siamese network is the basis for the object tracking module. The best model to solve the reidentification task is the Resnet50. For the Market 1501 dataset, it achieved Rank-1 efficiency of 84.6%. The system gives a visualization of the main paths of people's movements in the form of a heat map and assigns the direction where people most often look. In the experimental study, we assessed the system's effectiveness and time efficiency, and the current results give a perspective for its commercialization in the nearest future. • Development of the whole processing pipeline of human flow and reidentification. • Research aiming at the choice of competitive component methods • Experimental tuning of methods to achieve the high efficiency and low processing time. • Extension of the centroids algorithm with displacement vectors. • Platform for easy testing various methods of detection and reidentification. [ABSTRACT FROM AUTHOR]