1. Pose-guided multi-task video transformer for driver action recognition
- Author
-
Pizarro, Ricardo, Valle, Roberto, Bergasa, Luis Miguel, Buenaposada, José M., and Baumela, Luis
- Subjects
Computer Science - Computer Vision and Pattern Recognition - Abstract
We investigate the task of identifying situations of distracted driving through analysis of in-car videos. To tackle this challenge we introduce a multi-task video transformer that predicts both distracted actions and driver pose. Leveraging VideoMAEv2, a large pre-trained architecture, our approach incorporates semantic information from human keypoint locations to enhance action recognition and decrease computational overhead by minimizing the number of spatio-temporal tokens. By guiding token selection with pose and class information, we notably reduce the model's computational requirements while preserving the baseline accuracy. Our model surpasses existing state-of-the art results in driver action recognition while exhibiting superior efficiency compared to current video transformer-based approaches.
- Published
- 2024