<p>To address the issues of unstable temporal information and domain distribution disparity in facial expression, this paper proposes a Spatio-temporal Feature Point Attention Deep Transfer Network (SFPADTN) and uses it for cross-database facial expression recognition. Firstly, facial muscle movement information is represented by calculating the variation of spatio-temporal feature points and incorporated into facial expression. Stable temporal information is obtained by setting a threshold, and the spatio-temporal feature points are processed into an attention weight map, which is stacked with facial expression features to reflect the high attention regions during facial expression generation. Then, a deep transfer network achieves domain adaptation by minimising the maximum mean discrepancy objective function, and reduces the feature distribution disparity between different facial expression databases. Multiple transfer learning tasks are set up on the eNTERFACE, FABO, and RAVDESS database, and extensive cross-database facial expression recognition experiments are conducted. The experimental results validate the effectiveness of the proposed SFPADTN, which achieves accuracies of 52.42%, 44.44%, and 35.96% on the eNTERFACE, FABO, and RAVDESS databases, respectively, which are improvements of 1.19%, 1.91%, and 0.73% compared to the state-of-the-art methods in the same category.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Cross-database facial expression recognition based on Spatio-temporal Feature Point Attention Deep Transfer Network

  • Jingjie Yan,
  • Yuebo Yue,
  • Kai Yu,
  • Xiaoyang Zhou,
  • Ying Liu,
  • Jingsheng Wei

摘要

To address the issues of unstable temporal information and domain distribution disparity in facial expression, this paper proposes a Spatio-temporal Feature Point Attention Deep Transfer Network (SFPADTN) and uses it for cross-database facial expression recognition. Firstly, facial muscle movement information is represented by calculating the variation of spatio-temporal feature points and incorporated into facial expression. Stable temporal information is obtained by setting a threshold, and the spatio-temporal feature points are processed into an attention weight map, which is stacked with facial expression features to reflect the high attention regions during facial expression generation. Then, a deep transfer network achieves domain adaptation by minimising the maximum mean discrepancy objective function, and reduces the feature distribution disparity between different facial expression databases. Multiple transfer learning tasks are set up on the eNTERFACE, FABO, and RAVDESS database, and extensive cross-database facial expression recognition experiments are conducted. The experimental results validate the effectiveness of the proposed SFPADTN, which achieves accuracies of 52.42%, 44.44%, and 35.96% on the eNTERFACE, FABO, and RAVDESS databases, respectively, which are improvements of 1.19%, 1.91%, and 0.73% compared to the state-of-the-art methods in the same category.