Micro-expressions, fleeting facial movements lasting 1/25 to 1/5 of a second, are vital indicators of genuine emotions but are challenging to detect due to their subtlety and transience. This study proposes the Multi Action Unit Feature Fusion Network (MAUFFN), a novel deep learning framework to advance micro-expression recognition by addressing challenges such as subtle motion capture and class imbalance. MAUFFN comprises an Optical Flow Processing Module with an Optical Motion Amplification Module (OMAM), a MobileNetV3 Module, and a Cosine Fusion Module (CFM). The OMAM enhances subtle facial dynamics using the Flowmag model and TVL1 optical flow, enhancing motion feature extraction. The MobileNetV3 extracts robust features, while the CFM leverages Cosine Fusion Self-Attention to integrate local and global features via cosine similarity. Extensive evaluations using leave-one-subject-out cross-validation on CASME II, SAMM, SMIC-HS, and a composite dataset demonstrate MAUFFN’s superior performance, achieving a UF1 score of 85.28% and a UAR score of 87.53% UAR on the composite dataset, surpassing the leading prior method by 0.75% in UF1 and 0.65% in UAR.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi Action Unit Feature Fusion Network for Micro-Expression Recognition

  • Jiazheng Yang,
  • Kai Huang,
  • Xiaorui Zhu,
  • Heyou Chang,
  • Hao Zheng,
  • Jian Zhang

摘要

Micro-expressions, fleeting facial movements lasting 1/25 to 1/5 of a second, are vital indicators of genuine emotions but are challenging to detect due to their subtlety and transience. This study proposes the Multi Action Unit Feature Fusion Network (MAUFFN), a novel deep learning framework to advance micro-expression recognition by addressing challenges such as subtle motion capture and class imbalance. MAUFFN comprises an Optical Flow Processing Module with an Optical Motion Amplification Module (OMAM), a MobileNetV3 Module, and a Cosine Fusion Module (CFM). The OMAM enhances subtle facial dynamics using the Flowmag model and TVL1 optical flow, enhancing motion feature extraction. The MobileNetV3 extracts robust features, while the CFM leverages Cosine Fusion Self-Attention to integrate local and global features via cosine similarity. Extensive evaluations using leave-one-subject-out cross-validation on CASME II, SAMM, SMIC-HS, and a composite dataset demonstrate MAUFFN’s superior performance, achieving a UF1 score of 85.28% and a UAR score of 87.53% UAR on the composite dataset, surpassing the leading prior method by 0.75% in UF1 and 0.65% in UAR.