A Supervised Spatio-Temporal Contrastive Learning Framework with Optimal Skeleton Subgraph Topology for Human Action Recognition

Human action recognition (HAR) is a hotspot in the field of computer vision, the models based on Graph Convolutional Network (GCN) show great advantages in skeleton-based HAR. However,most existing GCN based methods do not consider the diversity of action trajectories, and not highlight the key joints. To address these issues, a supervised spatio-temporal contrastive learning framework with optimal skeleton subgraph topology for HAR (SSTCL-optSST) is proposed. SSTCL-optSST uses the samples with the same lablel as the target action (anchor) to build a positive sample set, each of them represents a trajectory of an action. The sample set is used to design a loss function to guide the model recognize different poses of the action. Furthermore, the subgraphs of an original skeleton graph are used to construct a skeleton subgraph topology space, each subgraph in it is evaluated, and the optimal one is selected to highlight the key joints. Extensive experiments have been conducted on NTU RGB+D 60 and Kinetics datasets, the results show that our model has competitive performance.