1. SiamEFT: adaptive-time feature extraction hybrid network for RGBE multi-domain object tracking
- Author
-
Shuqi Liu, Gang Wang, Yong Song, Jinxiang Huang, Yiqian Huang, Ya Zhou, and Shiqiang Wang
- Subjects
RGB and Event ,spatio-temporal ,hybrid network ,spiking neural networks ,neuromorphic computing ,object tracking ,Neurosciences. Biological psychiatry. Neuropsychiatry ,RC321-571 - Abstract
Integrating RGB and Event (RGBE) multi-domain information obtained by high-dynamic-range and temporal-resolution event cameras has been considered an effective scheme for robust object tracking. However, existing RGBE tracking methods have overlooked the unique spatio-temporal features over different domains, leading to object tracking failure and inefficiency, especally for objects against complex backgrounds. To address this problem, we propose a novel tracker based on adaptive-time feature extraction hybrid networks, namely Siamese Event Frame Tracker (SiamEFT), which focuses on the effective representation and utilization of the diverse spatio-temporal features of RGBE. We first design an adaptive-time attention module to aggregate event data into frames based on adaptive-time weights to enhance information representation. Subsequently, the SiamEF module and cross-network fusion module combining artificial neural networks and spiking neural networks hybrid network are designed to effectively extract and fuse the spatio-temporal features of RGBE. Extensive experiments on two RGBE datasets (VisEvent and COESOT) show that the SiamEFT achieves a success rate of 0.456 and 0.574, outperforming the state-of-the-art competing methods and exhibiting a 2.3-fold enhancement in efficiency. These results validate the superior accuracy and efficiency of SiamEFT in diverse and challenging scenes.
- Published
- 2024
- Full Text
- View/download PDF