Weakly Supervised Temporal Action Localization Through Segment Contrastive Learning

Weakly-supervised temporal action localization (WS-TAL) aims to learn localizing action with only video-level labels. In general, traditional methods process each snippet individually, thus ignoring both relationship between different action snippets and the productive temporal contextual relationships which are critical for action localization. In this paper, we propose Bidirectional Exponential Moving Average (BEMA) to utilize contextual information to obtain more stable feature representation. In addition to that, we introduce Inter-Segment Loss to refine the snippet representation in feature space to prevent misidentification of similar actions for accurate action classification, and Intra-Segment Loss to separate action from background in feature space to locate precise temporal boundaries. Substantial analyses show that our strategy is effective and brings comparable performance gain with current state-of-the-art WS-TAL methods on THUMOS’14 and ActivityNet v1.2 datasets. Our code can be found at https://github.com/JiangZhouHao/SCL .