STN-BA: Weakly-Supervised Few-Shot Temporal Action Localization

Existing Weakly-supervised Few-Shot Temporal Action Localization (WFTAL) methods often process feature snippets with limited information, resulting in prediction errors and poor localization performance. A novel model called Spatial-Temporal Attention Network with Boundary-check Algorithm (STN-BA) for WFTAL is proposed to address this issue. STN-BA enhances the discriminability of snippet features and has a particular fault tolerance mechanism. The proposed approach focuses on two main aspects: (1) a spatial-temporal attention module that establishes spatial-temporal relationships of action movement to enrich the feature information of each video snippet and (2) the implementation of a boundary-check algorithm to correct potential localization boundary errors. The network is trained to estimate Temporal Class Similarity Vectors (TCSVs) that measure the similarity between each snippet of untrimmed videos and reference samples. These TCSVs are then normalized and employed as a temporal attention mask to extract the video-level representation from untrimmed videos, enabling accurate action localization during testing. Experimental evaluations of the widely used THUMOS14 and ActivityNet1.2 datasets demonstrate that the proposed method outperforms state-of-the-art fully-supervised and weakly-supervised few-shot learning methods.