Spatiotemporal information complementary modeling and group relationship reasoning for group activity recognition

Exploring spatial-temporal interactions among group members is crucial for group activity recognition. However, most existing approaches cannot jointly consider it from multi-level cross-relations, which results in an incomplete representation. To address this issue, we propose a relational complementary module that comprehensively learns the interactions among members from both time-space and space-time perspectives. To suppress the information redundancy caused by this all-view interaction description, we introduce NH-Softmax to impose sparsity on the few relevant attention weights to generate robust and differentiated feature representations. In addition, to fully explore individual contextual interaction information, relaxed attention (RAT) is designed to enhance the feature information of each individual in a relaxed manner. It fleshes out individual representations by highlighting the most salient features and eases the computational burden. Our experiments on Volleyball dataset and Collective Activity dataset show significant improvements over previous state-of-the-art methods.