HARFMR: Human Activity Recognition with Feature Masking and Reconstruction

The widespread adoption of deep learning in the computer science field has significantly improved the functionality of wearable sensors, such as the recognition and localization of human activities. Nevertheless, the challenge of annotating and training sensor data persists due to the high associated costs. Unlabeled sensor data is more accessible and easier to train compared to labeled data, which has led to increased interest in self-supervised learning for human activity recognition. Masked reconstruction of raw sensor data is a method commonly employed in self-supervised learning. When applied to human activity recognition, the technique involves time-centric data masking and subsequent reconstruction. However, the masking and reconstruction of raw sensor data may potentially lead to the exclusion of crucial information, resulting in representations with lower semantic levels. To address this, we present a new strategy for masking and reconstruction, called Human Activity Recognition with Feature Masking and Reconstruction (HARFMR), specifically designed for human activity recognition. This architecture includes the masking of features using a random ratio and the subsequent reconstruction of the original sensor data, compelling the encoder to emphasize the contextual correlations of the data’s features and the properties of the features during the reconstruction process. Our evaluation of the proposed masking strategy on three public datasets demonstrates that the HARFMR method surpasses existing masking reconstruction schemes under self-supervised and semi-supervised settings.