Dual-branch vision transformer for low-resolution action recognition
摘要
The field of action recognition has primarily focused on identifying complex actions in high-quality videos, yet recognizing low-resolution movements remains a critical challenge with practical applications in real-world surveillance scenarios. This paper proposes a dual-branch vision transformer to address the spatiotemporal feature learning challenges in low-resolution action recognition. Our architecture employs distinct processing pathways: the primary branch captures spatial patterns using reduced frame rates, whereas the auxiliary branch operates at higher temporal resolution with spatial downsampling. Continuous feature interaction between branches is facilitated through cross-attention modules, enabling complementary spatiotemporal learning. This architectural separation allows targeted feature extraction: spatial analysis benefits from temporal context refinement while motion patterns gain enhanced spatial grounding. To validate our approach, we contributed a novel classroom behavior dataset containing 7,722 annotated videos (6,776 training / 1,546 tests) across 11 student action categories, captured through university surveillance systems. Experimental results demonstrate state-of-the-art performance with 75.20% F1-score on TinyVIRAT-v2 and 83.41% top-1 accuracy on our classroom dataset, outperforming existing baselines with only 13.68% additional computational overhead compared to single-branch transformers.