Enhancement of Masked Expression Recognition Inference via Fusion Segmentation and Classifier

Despite remarkable advancements in facial expression recognition, recognizing facial expressions from occluded facial images in real-world environments remains a challenging task. Various types of occlusions randomly occur on the face, obstructing relevant information and introducing unwanted interference. Moreover, occlusions can alter facial structures and expression patterns, causing variations in key facial landmarks. To address these challenges, we propose a novel approach called the Occlusion Removal and Information Capture (ORIC) network, which fuses segmentation and classification. Our method consists of three modules: the Occlusion Awareness (OA) module learns about occlusions, the Occlusion Purification (OP) module generates robust and multi-scale occlusion masks to purify the occluded facial expression information, and the Expression Information Capture (EIC) module extracts comprehensive and robust expression features using the purified facial information. ORIC can eliminate the interference caused by occlusion and utilize both local region information and global semantic information to achieve facial expression recognition under occluded conditions. Through experimental evaluations on synthetic occluded RAF-DB and AffectNet datasets, as well as a real occluded dataset FED-RO, our method demonstrates significant advantages and effectiveness. Our research provides an innovative solution for recognizing facial expressions from occluded facial images in wild environments.