Timeception Single Shot Action Detector: A Single-Stage Method for Temporal Action Detection

Temporal action detection is used to detect the start and end times and classify the potentially specific actions in a video. Prior studies in temporal action detection perform weak because they can not fully understand the whole input video's temporal structure and context information, and fail to adapt to the diversity of action time span. We propose a novel Timeception Single Shot Action Detector (TC-SSAD) to solve the problems mentioned above. In detail, we leverage the multiple Timeception layers to generate multi-scale feature sequences, where each Timeception layer uses depthwise-separable temporal convolution with multi-scale convolution kernels to capture the diversity of time spans. Besides, we use the super-event modules to learn the entire input video’s temporal structure and contextual information. The experimental results on THUMOS14 dataset show that when IoU threshold is 0.5, our method achieves 38.2% and 44.3% mAP on Two-stream features and Two-stream i3D features respectively, which is better than Decouple-SSAD network based method by 2.4% and 0.6%. Our method on Activitynet-1.3 dataset achieves 20.4% mAP, which is better than Decouple-SSAD network based method by 0.61% as far as Two-stream features on concerned.