Weakly-Supervised Temporal Action Localization by Background Suppression

We propose a novel method of background suppression to solve the issue that background regions are recognized as actions in weakly-supervised temporal action localization. The general attention-based action localization methods tend to use the attention module to generate segment-level attention weights. But there is little difference between the attentions from the background segments similar to the target actions and the attentions of the action segments, which causes the result that many background segments related to the target actions are still recognized as actions. To address this issue, a weakly-supervised temporal action localization network by background suppression (BS-WTAL) is designed. It introduces a filtering module for suppressing the background features and encouraging the action features, a classification module for identifying action categories and a generative attention module for segment-wise representation modeling. This enables BS-WTAL to suppress background to improve localization performance. Furthermore, we conduct ablation studies from different perspectives. Extensive experiments were performed on two datasets – THUMOS14 and ActivityNet1.2. Our approach shows better performance on these two datasets, even comparable with state-of-the-art fully-supervised methods.