Multimodal Contactless Human Stress Detection Using Deep Learning

It is widely acknowledged that everyone is subjected to some pressure or stress in his daily life. This is a feeling that always accompanies the person’s behavior and affects it negatively. For instance, driving a car under extreme stress can generally lead to a fatal accident. Today, the topic of automatic human stress detection is attracting an increasing number of researchers due to its great importance. The present article aims primarily to explore this subject using deep learning techniques with the purpose to detect and classify human stress and non-stress states. It has been revealed that the multimodal approach, which is based on data fusion, is a remarkably successful method. In this study, the fusion between facial expressions and physiological signals is used. The originality of the present study lies in the fact that the PPG signals used are remotely extracted from RGB facial videos. Moreover, the experimentation employs the UBFC-Phys dataset, a publicly available multimodal dataset. Different deep learning architectures, such as 3D-convolutional neural network (CNN) for the facial expressions and 1D-CNN for contactless PPG signals, are proposed in this article. The findings showed that the proposed methods can achieve validation accuracies as high as 83.33%, 75%, and 100% for facial expression, contactless PPG signals, and multimodal systems, respectively.