On Designing a Head Pose Estimation Approach in the Thermal Band Through GAN-Based Image Synthesis

Automated face biometric systems have a wide range of applications such as access control and secure banking. Head pose estimation (HPE) is one of the important modules of a face recognition system, which allows the system to determine if the pose in a face image is frontal. Head pose estimation is well studied in the visible spectrum, which makes this process trivial in applications such as identity verification. A contributing factor to the success of this research area is the availability of large-scale visible band face datasets. Unfortunately, the same is not true for the thermal band images, since there is a lack of thermal band face datasets, primarily due to the cost of the sensors. However, thermal spectrum imagery has applications in low-light and nighttime environments where visible band–based algorithms are not optimal. Therefore, algorithms tailored to the limited data and specificity of thermal spectrum face imagery must be studied to fill this research gap. To this end, our chapter presents an approach to synthesize visible band images from their thermal image counterparts to leverage visible band–based head pose estimators in thermal. We train a generative adversarial network to generate a visible face image from a thermal input and use pretrained visible-band pose estimators to estimate the yaw, pitch, and roll of the synthetic image. These estimations are then mapped back into the corresponding thermal face image. We show that our methodology outperforms a pose estimating model trained only on thermal data by decreasing the yaw, pitch, roll, and overall MAE by 26.8%, 42.5%, 36.3%, and 34.8%, respectively.