Back to Search
Start Over
Gesture Recognition with Keypoint and Radar Stream Fusion for Automated Vehicles
- Source :
- In Computer Vision-ECCV 2022 Workshops: Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part I (pp. 570-584). Cham: Springer Nature Switzerland
- Publication Year :
- 2023
-
Abstract
- We present a joint camera and radar approach to enable autonomous vehicles to understand and react to human gestures in everyday traffic. Initially, we process the radar data with a PointNet followed by a spatio-temporal multilayer perceptron (stMLP). Independently, the human body pose is extracted from the camera frame and processed with a separate stMLP network. We propose a fusion neural network for both modalities, including an auxiliary loss for each modality. In our experiments with a collected dataset, we show the advantages of gesture recognition with two modalities. Motivated by adverse weather conditions, we also demonstrate promising performance when one of the sensors lacks functionality.<br />Comment: Accepted for presentation at the 3rd AVVision Workshop at ECCV 2022, October 23, 2022, Tel Aviv, Israel
- Subjects :
- Computer Science - Computer Vision and Pattern Recognition
Subjects
Details
- Database :
- arXiv
- Journal :
- In Computer Vision-ECCV 2022 Workshops: Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part I (pp. 570-584). Cham: Springer Nature Switzerland
- Publication Type :
- Report
- Accession number :
- edsarx.2302.09998
- Document Type :
- Working Paper
- Full Text :
- https://doi.org/10.1007/978-3-031-25056-9_36