TBSA-Net: A Temperature-Based Structure-Aware Hand Pose Estimation Model in Infrared Images

In recent years, numerous researchers have conducted in-depth studies and made significant progress in 2D Hand Pose Estimation (HPE) tasks on RGB images. However, the field of HPE in the context of infrared images has received limited attention. Due to the limited channel information and high correlation with temperature, models designed for RGB images may suffer from insufficient accuracy in infrared images. Our experiments reveal that the temperature distributions of the human hand in infrared images exhibit significant regularity. In this paper, we propose the Temperature-Based Hand Judgement Model (TB-HJM) that leverages this characteristic. During the training phase, a higher penalty is given when the predicted pose’s temperature distribution does not align with the actual temperature distribution, and vice versa. In the testing phase, TB-HJM is utilized to select a hand proposal that closely matches the temperature distribution as the final output. Additionally, to address the lack of visual information in infrared images, we use PBNHead and GCN Refine Module to merge structural information into the network to ensure model accuracy. Experimental results demonstrate that our model outperforms the benchmark model (HRNet) by 1.72% in terms of AUC and achieves an improvement of 0.6448 by reducing the EPE from 3.02 to 2.38, achieving state-of-the-art performance on our infrared hand dataset.