Cross-Domain Facial Expression Recognition by Combining Transfer Learning and Face-Cycle Generative Adversarial Network

Facial expression recognition (FER) is one of the important research topics in computer vision. It is difficult to obtain high accuracy in FER tasks, especially when the high-quality labeled data are insufficient. Indeed, the facial images with non-frontal faces, occlusions and inaccurate labels heavily affects the training results of FER network models, which causes low recognition accuracy and poor robustness. To this end, we propose a novel strategy for FER tasks through combining transfer learning and generative adversarial network (GAN). First, we enlarge the training datasets by introducing an effective face-cycle GAN to synthesize additional facial expression images. Then, we develop two FER neural networks based on two representative convolutional neural networks (CNN). By transferring the cross-domain knowledge from the two well-trained CNNs to the proposed FER CNNs, it not only obtains more pre-trained knowledge and also accelerates the training process greatly. The experimental results show that the proposed FER CNNs integrated with the new face-cycle GAN achieves high accuracies 98.44%, 95.24% and 91.67% on three widely used datasets CK + , JAFFE, and Oulu-CASIA, respectively. Compared to the results obtained by other state-of-the-art FER methods, the accuracies are improved by 0.34%, 0.24%, and 2.62%, respectively.