Real-Time Emotion Recognition in Intelligent Tutoring Systems
摘要
This study investigates the integration of facial emotion recognition (FER) into Intelligent Tutoring Systems (ITS), with the aim of identifying the emotions that emerge throughout the learning process. Building on prior research, we argue that understanding students’ emotional states enables the delivery of more adaptive and effective feedback, thereby improving learning outcomes. A Deep Convolutional Neural Network (DCNN) was trained on the FER2013+ dataset, achieving a top-3 accuracy of 95.24% in classifying facial expressions across eight emotion categories. The model was integrated with MediaPipe to enable real-time emotion detection from video streams using a standard laptop camera, facilitating practical deployment in educational settings. Thirteen high school and early university students interacted with OATutor—an open-source ITS—while their facial expressions and on-screen activities were recorded. Emotional data from each frame was synchronized with an academic event log documenting actions such as starting a lesson, requesting help, or submitting answers. Results show that “surprise” was the most frequently observed emotion (over 85% of instances), whereas “anger,” “sadness,” and “contempt” appeared only in specific learning scenarios, particularly when students faced cognitive challenges or achieved multiple correct responses. Despite the absence of affective feedback from the system, students’ emotions fluctuated dynamically, suggesting active self-regulation processes. These findings demonstrate the feasibility of FER-enhanced ITS in real-world educational environments and underscore the need for future work integrating multimodal data and personalization strategies to optimize affective responsiveness in intelligent learning contexts.