Motion-Aware Topology Learning for Skeleton-Based Action Recognition

Graph convolutional networks have achieved great success in skeleton-based action recognition area, in which topology learning is the key component for extracting representative features. In this paper, we propose a novel module called Motion-Aware Topology Graph Convolution (MAT-GC) that can boost the graph modeling ability with motion-aware features and time-wise feature aggregation for skeleton-based action recognition. In particular, feature-level temporal differences of joints from adjacent frames are sampled and combined to form enhanced motion representations for a given frame. Moreover, a two-pathway structure is adopted to model pairwise correlations at each time step taking both short-term and long-term temporal motion variations into account. Eventually, joint features are aggregated following the time-wise topologies. Combined with temporal modeling modules, the overall graph convolutional network MAT-GCN is constructed. Experimental results on three popular skeleton-based action recognition datasets verify the effectiveness of the proposed method.