Human activity recognition using a bagging-based deep learning framework with convolutional capsule networks and spiking neural networks
摘要
This paper introduces a robust, multi-stage human activity recognition (HAR) system that can effectively recognize complicated human actions from video data. First, the raw video sequences are converted into frames, then applied median-filtering, and histogram-equalizing to minimize noise and normalize illumination. To enable reliable activity recognition, these frames converted to region of interest using canny edge detection. The system incorporates histogram of oriented gradients (HOG), improved scale-invariant feature transform (SIFT), speeded-up robust features (SURF), local binary patterns (LBP), and convolutional neural networks (CNNs) models for feature extraction, which provides spatial descriptions of each frame. To ensure model efficiency, correlation-based feature selection (CFS) is employed for reducing dimensionality and selects the appropriate features for classification. A unique two-tier classification method integrates convolutional capsule networks with attention (CONVCAPS-Attention) for spatial analysis and spiking neural networks (SNNs) for temporal dynamics, employing attention processes to improve feature significance. This robust HAR framework outperforms in terms of accuracy, precision, recall, and F1-score as compared with existing models.