CoT-Enhanced CNN-ViT Fusion with Adaptive Token Learning for Precise Facial Expression Recognition

Facial expression recognition is a popular research direction in computer vision. Traditional CNN models suffer from spatial localization problems and is difficult to learn the dependencies between different facial regions, thus limiting the global understanding of facial expressions. ViT performs excellently in image categorization and provides a new way of global information perception for the FER task, but lacks inductive bias and is prone to focus on noise. We propose an innovative CoT_FER hybrid architecture, which utilizes the CoT enhancement module to connect CNN and ViT, capturing local regional correlations to improve the global understanding of facial expressions, and adopts an adaptive learning method in the self-attention module of Transformer Encoder to dynamically retain tokens that are most relevant to the expression categorization, to overcome the problem of focusing noise and to improve the computational efficiency. Experiments show that the accuracy of the model on CK+, RAF-DB, and FERPlus is 99.2%, 91.07%, and 90.57%, respectively, demonstrating strong generalization ability and adaptability.