Spectrogram-Based Approach with Convolutional Neural Network for Human Activity Classification

Human activity recognition (HAR) is an expanding research field for analyzing holistic wellbeing trajectory, frailty detection and prevention of critical situations. With the increased availability of wearables and novel machine learning methods, the automatic recognition of human activities is exploited by real-time signals via Deep Learning techniques. This is due to their capability of learning contextual and localized patterns which give them a significant edge over traditional machine learning approaches. However, most of the state-of-the-art deep learning techniques have limitations due to limited number of features present in temporal dimension. In this regard, we propose Spectrogram-driven multilayer 2D-Convolutional Neural Network (2D-CNN) to classify among different types of human activities using triaxial accelerometer data obtained under MEDICON Scientific Challenge. The spectrogram has significant advantage over 1D time domain signals due to their capability to extract power spectrum in time as well as in frequency domain. The dataset consists of twelve activities of daily living and three types of simulated falls performed by subjects wearing a single accelerometer. In total, the dataset was composed by 468 instances. The spectrograms were determined by Short Time Fourier Transform (STFT) from the continuous signal obtained from X-, Y-, and Z-axis of the accelerometer signals. Experimental results show that our spectrogram driven 2D-CNN model reach an overall accuracy of 86.02% and an overall \({F}_{1}\) -score of 81.09% in classifying all the activity classes; significantly outperforming the deep learning architecture based on 1D time domain signal.