SHapley Additive exPlanations for Machine Emotion Intelligence in CNNs

With the rise of Artificial Intelligence (AI), the inclusion of emotion recognition in Machine Emotion Intelligence (MEI) has revolutionized Human–Computer Interaction (HCI). However, using Convolutional Neural Network (CNN) for emotion recognition presents a significant obstacle due to their inherent opacity. This lack of transparency and accountability poses a challenge, particularly in critical fields such as healthcare. This study proposes a new solution to a crucial problem by incorporating SHapley Additive exPlanations (SHAP) into CNNs. This research aims to develop transparent Machine Emotion Intelligence systems. It introduces SHAP, a practical interpretability method that reveals the hidden decision-making processes in CNNs. The study effectively connects model opacity and interpretability by using SHAP to shed light on specific facial features and their contributions to emotion classifications. Evaluating the performance and interpretability of three emotion detection models, namely ResNet50, AlexNet, and MobileNet, uncovers the nuanced trade-offs, with ResNet50 emerging as the best performer, achieving an impressive 86% test accuracy. SHAP interpretability analysis was then used on the models, offering critical insights and illuminating the importance of facial attributes in predictions, ultimately empowering the interpretation of the models.