Video understanding aims to empower computers to interpret video content in a manner similar to human comprehension. Although large-scale models achieve strong performance, their heavy computational and storage demands hinder real-time applications. To address this, we propose Motion-Enhanced Lightweight Network (MEL-Net), an efficient framework for action recognition with reduced complexity. MEL-Net introduces two fundamental modules: a Hybrid Distillation Module and a Motion Enhancement Module. The Hybrid Distillation Module combines feature and logits distillation, where feature distillation employs an Attention-Based Fusion (ABF) mechanism to aggregate multi-level student features, and logits distillation transfers soft-label “dark knowledge” from the teacher. The Motion Enhancement Module improves motion representation by extending the traditional two-channel motion vector (MV) with an additional norm channel and applying a Temporal Shift Module (TSM) to enrich temporal dynamics with minimal overhead. Experiments on HMDB51 and UCF101 demonstrate that MEL-Net achieves 71.7% and 91.3% accuracy, respectively, while reducing GFLOPs and parameters by 50% compared to models with similar accuracy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MEL-Net: Accelerating Action Recognition with Motion-Enhanced Lightweight Network

  • Die Tao,
  • JinXin Guo,
  • XinQi Li,
  • Ming Ma

摘要

Video understanding aims to empower computers to interpret video content in a manner similar to human comprehension. Although large-scale models achieve strong performance, their heavy computational and storage demands hinder real-time applications. To address this, we propose Motion-Enhanced Lightweight Network (MEL-Net), an efficient framework for action recognition with reduced complexity. MEL-Net introduces two fundamental modules: a Hybrid Distillation Module and a Motion Enhancement Module. The Hybrid Distillation Module combines feature and logits distillation, where feature distillation employs an Attention-Based Fusion (ABF) mechanism to aggregate multi-level student features, and logits distillation transfers soft-label “dark knowledge” from the teacher. The Motion Enhancement Module improves motion representation by extending the traditional two-channel motion vector (MV) with an additional norm channel and applying a Temporal Shift Module (TSM) to enrich temporal dynamics with minimal overhead. Experiments on HMDB51 and UCF101 demonstrate that MEL-Net achieves 71.7% and 91.3% accuracy, respectively, while reducing GFLOPs and parameters by 50% compared to models with similar accuracy.