Weighted TimeSformer Multi-region CNN Network for Personalized and Context-Aware Activity Localization in Ambient Assisted Living
摘要
Action localization, which entails the precise identification of both the type of activity and its corresponding temporal and spatial boundaries, is a fundamental task with significant applications in domains such as ambient assisted living and security surveillance. Although action localization has made great progress in the recent years, many of the existing systems still fail in addressing personalization and environmental context, which are some of the important elements in real-world applications e.g. ambient assisted living environments. Many often, the conventional approaches treat users and activities consistently, thereby missing out contextual cues such as environmental changes or object interactions thus neglecting individual variations in activity patterns. To this end, we propose a novel personalized and context-aware activity localization framework termed as Weighted TimeSformer Multi-Region CNN Network (WTMR-Net), catered for ambient assisted living system. By incorporating a context-aware Multi-Region CNN module along with the state-of-the-art TimeSformer framework, the proposed WTMR-Net combines the strengths of both convolutional and transformer-based models, enabling spatial feature extraction and long-range temporal modelling. Our method demonstrates superior performance on benchmark datasets including HMDB51, NTU RGB + D 60, and SVideoQA, consistently outperforming state-of-the-art approaches.