Reconstruction-driven contrastive learning for unsupervised skeleton-based human action recognition

At present, researchers intend to use unlabeled skeleton data for human action recognition considering the cumbersome process of annotating large-scale datasets. Therefore, how to learn discriminate human action representation in an unsupervised manner is essential to achieve high recognition accuracy. In this paper, a novel framework called reconstruction-driven contrastive learning named RdCL to obtain unsupervised action representation is proposed. First, we utilize several data augmentation methods to generate similar positive skeleton samples. Second, proposed RdCL is applied to learn action representation. In RdCL, the learned representation by contrastive learning is simultaneously used to reconstruct skeleton sequence through a strong encoder and a weak decoder. The reconstruction learning of skeleton guides the contrastive learning to concentrate on detailed joint coordinates, which helps to learn more discriminative action features. Finally, the K-nearest neighbor classifier with learned representation is applied for action classification. Recognition comparisons with some unsupervised and supervised methods are given on datasets of five different scales. Especially, we perform extensive experiments on large-scale datasets like PKU-MMD, NTU RGB+D, and NTU RGB+D 120 to show the efficiency of our work.