AU-Oriented Expression Decomposition Learning for Facial Expression Recognition

Facial Expression Recognition (FER) has received extensive attention in recent years. Due to the strong similarity between expressions, it is urgent to distinguish them meticulously in a finer-grained manner. In this paper, we propose a method, named AU-oriented Expression Decomposition Learning (AEDL), which aims to decouple expressions into Action Units (AUs) and focuses on subtle facial differences. In particular, AEDL comprises two branches: the AU Auxiliary (AUA) branch and the FER branch. For the former, the generic knowledge of dependencies among AUs is leveraged to supervise AU predictions which are then transformed into new expression predictions with a learnable matrix modeled by the relationship between AUs and expressions. For the latter, fusion features are employed to compensate for the minority classes to ensure adequate feature learning. FER predictions are guided by the AUA branch, mining detailed distinctions between expressions. Importantly, the proposed method is independent of the backbone network and brings no extra burden on inference. We conduct experiments on popular in-the-wild datasets and achieve leading performance, proving the effectiveness of the proposed AEDL.