Macro and Micro-Expressions Spotting Based on Cross-Attention Transformer Progressive Fusion Network
摘要
Micro-expression (ME), as a specific spontaneous and unconscious facial expression, reveals human genuine thoughts and intentions. Due to the brief duration and subtle amplitude, it is extraordinarily challengeable to spot ME from an ordinary long video, especially confused with the Macro-expression (MaE). To address this issue, this paper proposes a Multi-level Cross-attention Swin Transformer Fusion model for diverse detail feature representation on ME spotting, namely MCSTF. Specifically, this model first crops and aligns the input video frames, then yields optical flow patterns between two adjacent frames, concentrating on specific facial regions with the Region of Interest (ROI) operator. Subsequently, these preprocessed features are separately fed into the multi-stream convolutional neural network channel and the multi-level channel for diverse subtle features extraction. The cross-attention mechanism is embedded based on the Swin Transformer to facilitate local feature interaction for cross-channel relations. Moreover, the hierarchical maps from different stages are progressively aggregated to enhance the global representation on ME. In the end, the output component of the MCSTF model is to predict the probability of a particular frame corresponding to a specific expression. Extensive comparison experiments show that the proposed MCSTF model achieves the efficient local and global feature fusion of hierarchical optical flow features, which outperforms other ME spotting methods based on deep neural networks.