Periodic-Aware Network for Fine-Grained Action Recognition

Recently, skeleton-based action recognition has gained increasing attention and achieved remarkable results in coarse-grained action recognition. Despite the positive results shown in these attempts, they are less effective in scenarios that require a detailed comparison between fine-grained classes, e.g. different moves during a vault. In such scenarios, existing methods make it hard to distinguish subtle differences between actions with different numbers of repetitions. In this article, to solve the above problem, we introduce periodicity into fine-grained action classification and propose a novel network architecture named periodic-aware network (PAN) to distinguish fine-grained actions with different numbers of repetitions. Firstly, a periodicity feature extraction module (PFEM) is proposed to capture periodicity information and extract periodicity features of different levels. Then, a periodicity fusion module (PFM) is proposed to fuse periodicity features and spatiotemporal features. We apply multiple periodicity fusion modules to fuse different levels of features. Finally, the results are obtained by classifying the fusion features. Extensive experiments on two fine-grained skeleton-based action recognition datasets, namely FineGym and Diving48, show that our proposed method outperforms previous skeleton-based action recognition methods.