Enhancing facial action unit intensity estimation with ordinal regression-enhanced transformer

Facial action unit (AU) intensity estimation from static images presents a significant challenge in computer vision due to the absence of dynamic visual cues. This paper introduces a novel method, the ordinal regression-enhanced AU intensity transformer (OR-AUIT), which integrates ordinal information as temporal context to enhance AU intensity estimation. OR-AUIT employs a Siamese vision transformer to learn ordinal relationships between paired samples within facial expression sequences and estimate AU intensities in a multitask learning framework. Experimental results on the DISFA and FERA2015 datasets demonstrate state-of-the-art performance, with an ICC of 0.696 and MAE of 0.157 on DISFA, and an ICC of 0.751 and MAE of 0.454 on FERA2015, showcasing the method’s effectiveness in leveraging temporal context for improved AU intensity estimation. The source code will be publicly released at our GitHub repository: https://github.com/shiysu231-glitch/OR-AUIT.