Scholarlink

<p>Classroom behavior analysis plays a crucial role in understanding student engagement, learning effectiveness, and instructional quality in higher education.This study presents a multimodal deep learning framework for classifying students’ classroom behaviors by integrating visual and physiological data to support data-driven instruction. To address the limitations of existing approaches—namely complex classroom settings, large data volumes, and reliance on single modalities—we develop a Multi-Level Cross-Self Attention Fusion Network (MCSFN). The network comprises three branches for extracting spatial visual features, Restormer-enhanced video representations, and physiological signals modeled via a SiLU-activated multilayer perceptron. These heterogeneous features are fused through a multi-level cross-attention mechanism for robust representation learning. Evalua ted on a self-constructed dataset of classroom videos and wearable-derived signals, the proposed framework achieves 95.88% classification accuracy, surpassing single-modality baselines. Beyond improving behavioral recognition accuracy, this work highlights the potential of multimodal AI-driven analytics to inform instructional design, enable real-time monitoring, and enhance personalized learning interventions, offering a practical pathway toward data-informed and adaptive classroom practices.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing classroom behavior analysis with multimodal data: a cross-attention fusion network approach

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月4日

查看全文
  • Bo Li,
  • Qi Zhao,
  • Mengmeng Wei,
  • Xu Yang,
  • Xiaofeng An

摘要

Classroom behavior analysis plays a crucial role in understanding student engagement, learning effectiveness, and instructional quality in higher education.This study presents a multimodal deep learning framework for classifying students’ classroom behaviors by integrating visual and physiological data to support data-driven instruction. To address the limitations of existing approaches—namely complex classroom settings, large data volumes, and reliance on single modalities—we develop a Multi-Level Cross-Self Attention Fusion Network (MCSFN). The network comprises three branches for extracting spatial visual features, Restormer-enhanced video representations, and physiological signals modeled via a SiLU-activated multilayer perceptron. These heterogeneous features are fused through a multi-level cross-attention mechanism for robust representation learning. Evalua ted on a self-constructed dataset of classroom videos and wearable-derived signals, the proposed framework achieves 95.88% classification accuracy, surpassing single-modality baselines. Beyond improving behavioral recognition accuracy, this work highlights the potential of multimodal AI-driven analytics to inform instructional design, enable real-time monitoring, and enhance personalized learning interventions, offering a practical pathway toward data-informed and adaptive classroom practices.