Multiple Attention Network for Facial Expression Recognition

Facial expression recognition (FER) has become increasingly important in the field of human-computer interaction. This paper proposes an improved method with attention mechanism to improve FER performance. Our approach is grounded in two crucial observations. Firstly, as multiple categories share similar underlying facial characteristics, distinguishing between them may be subtle. Secondly, recognizing facial expressions demands a comprehensive approach by encoding high-order interactions among localized features from multiple facial regions simultaneously. To address these challenges, we introduce our MAN model consisting of three key components: Multi-branch stack Residual Network (MRN), Transitional Attention Network (TAN), and Appropriate Cascade Structure (ACS). The TAN learns objectives to maximize class separability, while the MRN deploys attention heads to focus on various facial regions and generate attention maps. Additionally, the ACS module provides a more reasonable construction method for the model. Comprehensive experiments on three publicly available datasets (AffectNet, RAF-DB, and CK+) consistently achieves better expression recognition performance. Compared to the ResNet network, our approach yielded an improved accuracy of 3.3%, 3.2%, 4.1% and 1.8% on the Affectnet7 dataset, Affectnet8 dataset, RAF-DB dataset, and CK+ dataset, respectively.