The Face Behind the Mask: Thermography of the Face

What can facial thermography tell us about emotions? The notion that we can leverage measures of bodily responses to reveal something about our inner emotional experiences harks back to William James’ ground-breaking theory on the nature of emotions (James, 1884). At the same time, this quest has met with many criticisms along the way (see Cannon, 1927; Levenson, 2003). Deciphering the relationships between bodily signals and human emotions has fascinated generations of scholars (Levenson, 2003). The face, in particular, continues to attract a lot of interest (Kappas et al., 2013), as researchers hope that subtle and almost invisible changes in facial expressions might provide a key to unveiling someone’s genuine emotions. Where only a few specifically trained experts were previously believed to be able to see and interpret subtle cues of deception in facial muscle movements (Ekman et al., 1999; Frank & Svetieva, 2015), recent technical advances in computational imaging (Nowara et al., 2022) and machine learning may be on the verge of changing this picture (Bian et al., 2024). Perhaps even more stunning, however, is the flexibility with which machine learning methods are beginning to integrate biosignals from different modalities and devices. For example, traditional RGB image data can now be fused with data from the invisible thermal spectrum (Chen et al., 2018; Wang et al., 2014, 2018), thereby allowing researchers to indeed look beneath the skin of facial movements. Could this mean that we may be close to the holy grail of reliably revealing the truth about someone’s emotions and thus revealing the face behind the mask?