Joint Relation Modeling and Feature Learning for Class-Incremental Facial Expression Recognition

Due to the diversity of human emotions, it is often difficult to collect all the expression categories at once in many practical applications. In this paper, we investigate facial expression recognition (FER) under the class-incremental learning (CIL) paradigm, where we define easily-accessible basic expressions as an initial task and learn new compound expressions continuously. To this end, we propose a novel joint relation modeling and feature learning (JRF) method, which mainly consists of a local nets module (LNets), a dynamic relation modeling module (DRM), and an adaptive feature learning module (AFL) by taking advantage of the relationship between old and new expressions, effectively alleviating the stability-plasticity dilemma. Specifically, we develop LNets to capture subtle distinctions across expressions, where a novel diversity loss is designed to locate informative facial regions in each local net. Then, we introduce DRM to enhance feature representations based on two types of graph convolutional networks (GCNs) (including an image-shared GCN and two image-specific GCNs) from the perspectives of global-local graphs and old-new classes. Finally, we design AFL to explicitly fuse old and new class features via a weight selection mechanism. Extensive experiments on both in-the-lab and in-the-wild facial expression databases demonstrate the superiority of our method in comparison with several state-of-the-art methods for class-incremental FER.