Human Action Recognition with Infrared Images Based on Dual-pathway Trans-ResNet Model
摘要
Human action recognition (HAR) holds a significant research significance and a widespread application value. Due to the characteristics of infrared information, such as minimal susceptibility to lighting effects and privacy protection, methods for HAR based on infrared information have garnered considerable attention. However, existing research methods fail to simultaneously address both the local contour features and global temporal features of human actions. Therefore, this paper proposes a dual-pathway Trans-ResNet model for HAR with infrared images, leveraging the local detail extraction capability of the ResNet18 of convolutional neural networks (CNNs) and the global information extraction capability of the Transformer architecture. Firstly, we construct convolutional groups to enhance the model's nonlinear expression capability. Secondly, we adopt the idea of channel separation to divide the feature maps of intermediate layers into two parts in the channel dimension, which are fed into the CNN path and Transformer path of the Trans-ResNet, respectively. Additionally, a perceptual channel enhanced attention module is introduced into the Transformer path to enhance its receptive field. Finally, we utilize residual multi-layer perceptron modules to fully fuse the features of these two paths. Due to the extreme limitation of openly accessible datasets, we construct an infrared image dataset by performing continuous frame concatenation on infrared images containing seven action categories. Experimental results demonstrate that the dual-pathway Trans-ResNet model not only has fewer parameters but also achieves a recognition accuracy of 99.49% for human actions in the test set, surpassing other models and methods based on infrared images.