Dynamic-Static Graph Convolutional Network for Video-Based Facial Expression Recognition

Most of the current methods for video-based facial expression recognition (FER) in the wild are based on deep neural networks with attention mechanism to capture the relationships between frames. However, these methods suffer from the large variations of expression patterns and data uncertainties. This paper proposes a Dynamic-Static Graph Convolutional Network (DSGCN), which mainly consists of a Static-Relational graph (SRG) and a Dynamic-Relational graph (DRG). The SRG aims to guide the network to learn the static spatial relationship of facial expressions in each video frame, strengthening the salient areas of the face through the dependencies of context nodes. The DRG learns the dynamic temporal relationship of facial expressions by aggregating video sequence features, constructing a graph with other samples within a batch to share facial expression features with different contexts, thus promoting feature diversity to improve robustness. The proposed DSGCN framework achieves state-of-the-art results on the FERV39K, DFEW and AFEW benchmarks, and ablation experiments verify the effectiveness of each module.