A deep convolutional attention network based on RGB activity images for smart home activity recognition

Human activity recognition (HAR) serves as a fundamental component for the various applications of smart homes. However, although existing methods can extract temporal features of sensor event sequences, they do not consider the activation time, location specificity, and long-term dependence of sensor events. This article proposes a new HAR approach based on RGB activity images and a deep convolutional attention network (DCAN). First, the raw data is segmented according to the activity labels after preprocessing. Subsequently, a sliding window is utilized to partition the segmented activity instances into several fixed-length parts. Next, the windowed part is converted into an RGB activity image, where the x-coordinate represents the event sequence, the y-coordinate represents the sensor pattern, and the pixel value is a triplet composed of the event’s start time, end time, and duration. Then DCAN is used to extract features and classify these RGB activity images. We test the proposed method on the Aruba dataset and the weighted F1 scores of 10-fold cross-validation for 10 and 8 activities are 0.953 and 0.992, respectively, indicating that our proposed RGB activity image effectively improves classification and our DCAN classifier outperforms existing methods.