Evaluation Method of Online Education Learners’ Emotional Input Based on Multimodal Data Fusion

In order to better understand learners’ emotional orientation and learning status, an online education learners’ emotional engagement assessment method based on multimodal data fusion was proposed. In order to comprehensively evaluate learners’ emotional engagement, two modal data, comment data and facial expression images, were selected as the evaluation basis. Collect comment data using crawler technology and preprocess word segmentation and part of speech tagging; Extract features from comment data using improved TF-IDF and construct a classifier for comment data using the K-nearest neighbor algorithm. Using a camera to capture facial expression images, and performing lighting transformation, graying, and filtering; Extract HOG features of facial expression images using absolute gradient histogram algorithm, and construct a classifier for facial expression images using Adaboost algorithm. By synthesizing the above two parts of the process, two evaluation results were obtained, with different weights set for each type of single modal data. The weighted sum rule is used to fuse multimodal data at the decision-making level to obtain the final evaluation decision result. The results show that the MSE, MAE and MAPE of the evaluation methods based on review text, facial expression and body movements are relatively smaller than those based on body movements, which indicates that the evaluation accuracy is higher.