Recognizing autism-related behaviors from real-world data is a crucial step toward supporting early diagnosis and intervention. However, existing methods are often limited to single-modality inputs or rely on computationally intensive multimodal architectures, hindering practical deployment. To address these challenges, this paper presents a lightweight audio-visual framework based on the AV-ASD dataset. The approach combines TimeSformer for efficient video-based spatio-temporal representation and Wav2Vec2 for audio encoding, integrated via a Transformer-based fusion module to capture temporal and cross-modal dependencies. The proposed model achieves strong performance across key behavior categories, demonstrating the potential of efficient multimodal learning for autism behavior analysis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Efficient Lightweight Multimodal Framework for Autism Behavior Recognition on the AV-ASD Dataset

  • Van-Minh Luong,
  • Truc Nguyen

摘要

Recognizing autism-related behaviors from real-world data is a crucial step toward supporting early diagnosis and intervention. However, existing methods are often limited to single-modality inputs or rely on computationally intensive multimodal architectures, hindering practical deployment. To address these challenges, this paper presents a lightweight audio-visual framework based on the AV-ASD dataset. The approach combines TimeSformer for efficient video-based spatio-temporal representation and Wav2Vec2 for audio encoding, integrated via a Transformer-based fusion module to capture temporal and cross-modal dependencies. The proposed model achieves strong performance across key behavior categories, demonstrating the potential of efficient multimodal learning for autism behavior analysis.