Facial expression intensity estimation using label-distribution-learning-enhanced ordinal regression

Facial expression intensity estimation has promising applications in health care and affective computing, such as monitoring patients’ pain feelings. However, labeling facial expression intensity is a specialized and time-consuming task. Ordinal regression (OR)-based methods address this issue to some extent by estimating the relative intensity but failing to estimate the absolute intensity due to lack of exploring useful information from noisy labels caused by manual and automatic labeling biases. Inspired by label distribution learning (LDL) to resist the noisy labels, this paper introduces the label-distribution-learning-enhanced OR (LDL-EOR) approach for facial expression intensity estimation. This design aims to utilize LDL to improve the accuracy of absolute intensity estimation while keeping the cost of manual labeling low. The label distribution is converted into a continuous intensity value by calculating the mathematical expectation, which makes the prediction results meet both relative and absolute intensity constraints. Ensuring the feasibility of LDL-EOR in different supervised settings, this paper presents a unified label distribution generation framework to automatically relabel training data frame by frame. The generated soft labels are used to supervise the LDL-EOR model and enhance its robustness to the noise existing in the original labels. Numerous experiments were conducted on three public expression datasets (CK+, BU-4DFE, and PAIN) to validate the superiority of LDL-EOR relative to other state-of-the-art approaches.