Hybrid 2D–3D CNN with Feature Fusion and Spatial–Temporal Attention for Violence Detection
摘要
Detecting violence in videos is an important task for ensuring public safety and security. It can be difficult, as it involves analyzing videos from multiple surveillance cameras at any time and place. The system should detect violent activities accurately and quickly, alerting security personnel in real time. To accomplish this, we propose a hybrid approach that combines the strengths of 2D convolutional neural networks (2D CNNs) and 3D CNNs for violence detection. Our approach also incorporates feature fusion, spatial and temporal attention mechanisms to improve the model’s accuracy. Specifically, we first use a 3D convolution layer to capture temporal features by considering the sequence of frames over time. Then, we use a 2D depth-wise convolution layer to extract spatial features from each video frame and create spatial attention. The spatial attention module is called depth-wise spatial attention (DSA). We also exploit a simple channel attention module to focus on temporal features. While the spatial and temporal attention modules are applied to the input videos, we fuse these extracted features using a proposed feature fusion method named max-pooling fusion module (MFM). Since the proposed framework relies on deep and complex modules, it requires the computational power provided by high-performance computing (HPC) systems. To evaluate our approach, it is applied to four benchmark datasets. Our experiments demonstrate that the proposed hybrid approach achieves superior performance compared to several state-of-the-art methods, making it an effective solution for violence detection in videos.