Deep Learning Based Multimodal Human Activity Recognition in Smart Homes Using SDHAR-HOME

This paper presents a comparative analysis of two deep learning models, Bidirectional Recurrent Neural Networks (BiRNN) and a Self-Attention-based model, for multimodal Human Activity Recognition (HAR) using the SDHAR-HOME dataset. This real-world dataset captures diverse, non-intrusive sensor data, including motion, contact, temperature, humidity, vibration, ambient light, Bluetooth-based indoor positioning, and smart plug electricity consumption. By integrating appliance-level energy usage as a behavioral signal, the study introduces a novel privacy-conscious approach to activity recognition, offering an alternative to intrusive vision and audio-based systems. To classify complex and asynchronous activities, BiRNN is employed to capture bidirectional temporal dependencies, while a transformer-inspired Self-Attention model is implemented to attend selectively to the most informative temporal and sensor-specific features. The Self-Attention model outperforms BiRNN in weighted-average precision (User-1: 0.90; User 2: 0.82), recall (User-1: 0.77; User 2: 0.75), and F1-score (User-1: 0.83; User 2: 0.80), indicating a superior capability in classifying diverse activities. In contrast, the BiRNN model shows lower performance, particularly in weighted-average recall (User-1: 0.69; User 2: 0.70), and F1-score (User-1: 0.74; User 2: 0.73), due to its limited capacity to model long-range dependencies. This study introduces a benchmark framework that combines electric power data with environmental signals, contributing to the design of ethical, scalable HAR systems that respect user privacy, support aging-in-place, and enable equitable AI integration. The findings underscore that even limited appliance-level consumption data can be a valuable feature in multimodal HAR pipelines. By leveraging both temporal modeling and attention mechanisms, this work advances the development of responsible and privacy-aware activity recognition systems for smart home applications.