Enhancing Emotion Detection Accuracy and Transparency Through Multimodal Fusion and Explainable AI
摘要
The proposed research creates a state-of-the-art multimodal emotion detector that applies Explainable AI (XAI) approaches for precision enhancement and aspect visualization. The dual analysis of text and audio, and video data leads to human emotion classification through XAI methods, including LIME, SHAP, and Grad-CAM that produce explanations for predictions. Additional training data derived from these explanations helps the individual models become more reliable and produces results with greater interpretability. The refined models will combine through multimodal fusion by using averaging methods or concatenation to develop a precise ethical operator of emotion detection. The proposed study focuses on resolving black-box model issues while enhancing interpretability features and decreasing biases that affect fair decision processes across applications handling mental health evaluations and human–machine interaction, and customer support activities.