Action localization, which entails the precise identification of both the type of activity and its corresponding temporal and spatial boundaries, is a fundamental task with significant applications in domains such as ambient assisted living and security surveillance. Although action localization has made great progress in the recent years, many of the existing systems still fail in addressing personalization and environmental context, which are some of the important elements in real-world applications e.g. ambient assisted living environments. Many often, the conventional approaches treat users and activities consistently, thereby missing out contextual cues such as environmental changes or object interactions thus neglecting individual variations in activity patterns. To this end, we propose a novel personalized and context-aware activity localization framework termed as Weighted TimeSformer Multi-Region CNN Network (WTMR-Net), catered for ambient assisted living system. By incorporating a context-aware Multi-Region CNN module along with the state-of-the-art TimeSformer framework, the proposed WTMR-Net combines the strengths of both convolutional and transformer-based models, enabling spatial feature extraction and long-range temporal modelling. Our method demonstrates superior performance on benchmark datasets including HMDB51, NTU RGB + D 60, and SVideoQA, consistently outperforming state-of-the-art approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Weighted TimeSformer Multi-region CNN Network for Personalized and Context-Aware Activity Localization in Ambient Assisted Living

  • Sravan Kumar Nakerakanti,
  • Gokul Krishna Reddy Chiraiahgari,
  • Athira Nambiar

摘要

Action localization, which entails the precise identification of both the type of activity and its corresponding temporal and spatial boundaries, is a fundamental task with significant applications in domains such as ambient assisted living and security surveillance. Although action localization has made great progress in the recent years, many of the existing systems still fail in addressing personalization and environmental context, which are some of the important elements in real-world applications e.g. ambient assisted living environments. Many often, the conventional approaches treat users and activities consistently, thereby missing out contextual cues such as environmental changes or object interactions thus neglecting individual variations in activity patterns. To this end, we propose a novel personalized and context-aware activity localization framework termed as Weighted TimeSformer Multi-Region CNN Network (WTMR-Net), catered for ambient assisted living system. By incorporating a context-aware Multi-Region CNN module along with the state-of-the-art TimeSformer framework, the proposed WTMR-Net combines the strengths of both convolutional and transformer-based models, enabling spatial feature extraction and long-range temporal modelling. Our method demonstrates superior performance on benchmark datasets including HMDB51, NTU RGB + D 60, and SVideoQA, consistently outperforming state-of-the-art approaches.