Integration of machine learning bi-modal engagement emotion detection model to self-reporting for educational satisfaction measurement

In line with the Saudi Arabia Vision Re- alization Program 2030, improving the quality of life of citizens through increased recreational opportunities and optimized educational time has become a top pri-ority. Consequently, a wealth of self-report tools has been developed to evaluate quality of life. Our study contributes to resolving the students’ self-report bias issue by introducing emotion detection as an additional factor to self-report in the measurement of educational satisfaction. The study first provides a brief introduction to automatic emotion detection systems and then presents its contribution to the integration of self-report and emotion detection. The proposed method for detecting emotions utilizes a machine learning-based bi- modal approach that involves facial expression and body gestures, utilizing an Ensemble Machine Learning (EM) model that incorporates a Support Vector Machines Classifier (SVM), Random Forest Classifier (RF), and Logistic Regression Classifier (LR). The emotions detector achieved an accuracy of 97% on the Emotions in Context Dataset (EMOTIC). This study highlights the vital significance of emotional experience measurement in educational sessions as well as the potential pitfalls of self-reporting. Therefore, it can be utilized as a performance metric for educational programs within the Saudi Quality of Life program.