Search

Understanding actions in videos remains a significant challenge in computer vision, which has been the subject of several pieces of research in the last decades. Convolutional neural networks (CNN) are a significant component of this topic and play a crucial role in the renown of Deep Learning. Inspired by the human vision system, CNN has been applied to visual data exploitation and has solved various challenges in various computer vision tasks and video/image analysis, including action recognition (AR). However, not long ago, along with the achievement of the transformer in natural language processing (NLP), it began to set new trends in vision tasks, which has created a discussion around whether the Vision Transformer models (ViT) will replace CNN in action recognition in video clips. This paper conducts this trending topic in detail, the study of CNN and Transformer for Action Recognition separately and a comparative study of the accuracy-complexity trade-off. Finally, based on the performance analysis's outcome, the question of whether CNN or Vision Transformers will win the race will be discussed. [ABSTRACT FROM AUTHOR]

This paper proposed a novel pretext task to address the skeleton-based video representation for self-supervised action recognition tasks. Instead of exploiting only the whole body, various levels of the skeleton structure (e.g., upper body, lower body, left arm, left leg, right arm, right leg, and torso) are employed to extract essential coarser-grained characteristics. This involves computing statistical representations like motion, orientation, trajectory, and magnitude shift from unlabeled skeleton configurations. Then a learning model is built and trained to yield these statistical representations given the sequence configuration as the input. Our approach is question-driven, where each question acts as a puzzle piece contributing to a deeper understanding of the skeleton joint configuration. It's inspired by the ability of the cognitive system observed in individuals to hypothesize unseen actions. This is accomplished by posing pertinent questions and envisioning plausible scenarios to recognize the actions taking place. The answers to these devised questions are derived from the statistical representation of skeleton configurations. To this end, we made 44 questions designed to encompass the broadest overview to the finest detail. Our experiments on the NTU RGB-D, NW-UCLA, and PKU-MMD datasets demonstrate outstanding results in action recognition, proving the superiority of our approach in learning discriminative characteristics. [ABSTRACT FROM AUTHOR]