A Multibranch LSTM-CNN Model for Human Activity Recognition

The application of various deep learning methods in the field of Human Activity Recognition (HAR) is becoming popular day by day. Although many works in the past depended on pre-defined, manually engineered features for classification, deep learning has shown great potential in this field recently. Specifically, deep learning models based on Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) layers have outperformed other models. This work proposes a multi-branch CNN-LSTM model with two branches, one of which is an LSTM branch and the other one is a CNN branch. Features are extracted individually by the two branches, followed by a decision-making process based on concatenated features of two branches using an additional LSTM layer. Overfitting can be a big issue in deep learning models, especially when the dataset is imbalanced as the model becomes biased towards the majority class. This work also investigated the performance of models after data balancing with synthetic signals. The performance of the model was evaluated on three widely used public datasets in the field (UCI HAR dataset, WISDM dataset, and PAMAP2 dataset). The proposed model obtained F1 scores of 96.435, 95.89, and 89.94% on the datasets respectively. The results show that the proposed model outperformed many existing approaches on the same datasets with fewer parameters.