Scholarlink

With the development of mobile technology, the variety of sensors in portable devices continues to increase, drawing significant attention to human activity recognition (HAR) based on sensors. However, this field faces challenges such as insufficient feature extraction and weak correlation of heterogeneous data. To address this, we propose a self-supervised multimodal fusion network (SSMF), which integrates frequency domain data while retaining the time domain data of accelerometers and gyroscopes to enrich the classification basis. Additionally, we employ a Transformer module for feature fusion to enhance correlation and utilize a self-supervised learning (SSL) strategy to assist training and strengthen feature interaction. Experimental results show that SSMF outperforms other comparison methods on three benchmark datasets. Our code is available at https://github.com/bx12138/SSMF .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FreqTime-HAR: Self-supervised Multimodal Fusion via Transformer for Robust Human Activity Recognition

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Xiang Wu,
  • Bin Zhao

摘要

With the development of mobile technology, the variety of sensors in portable devices continues to increase, drawing significant attention to human activity recognition (HAR) based on sensors. However, this field faces challenges such as insufficient feature extraction and weak correlation of heterogeneous data. To address this, we propose a self-supervised multimodal fusion network (SSMF), which integrates frequency domain data while retaining the time domain data of accelerometers and gyroscopes to enrich the classification basis. Additionally, we employ a Transformer module for feature fusion to enhance correlation and utilize a self-supervised learning (SSL) strategy to assist training and strengthen feature interaction. Experimental results show that SSMF outperforms other comparison methods on three benchmark datasets. Our code is available at https://github.com/bx12138/SSMF .