Online learners’ engagement detection via facial emotion recognition in online learning context using hybrid classification model

Writing, reading, viewing video lectures, completing online examinations, and attending online meetings are all activities that students participate in through the internet. While participating in these educational activities, they demonstrate various degrees of interest, including boredom, aggravation, delight, indifference, confusion, and learning gain. Online educators must accurately and efficiently monitor the degree of engagement of their online learners with the goal of giving focused pedagogical assistance to them through interventions. The objective of this paper is to propose a novel students engagement prediction model for online learners based on facial emotion, which will include four basic phases: (a) preprocessing, (b) feature extraction, (c) emotion recognition, and (d) student engagement prediction. The preprocessing step is the first phase in which the Face detection process is followed. Following the preprocessing step, the feature extraction phase proceeds with the extraction of the Improved Active Appearance Model (AAM), Shape Local Binary Texture (SLBT), Global Binary Pattern (GBP), and ResNet features. The retrieved characteristics are subsequently subjected to emotion recognition via the Hybrid Classification model, which incorporates models including Improved Deep Belief Network (IDBN) and Convolutional Neural Network (CNN). The student's involvement or engagement is identified based on the emotions recognized, as well as the way they performed via the enhanced entropy-based process. The execution of the suggested hybrid IDBN + CNN model is evaluated over the extant methods like DBN, SVM, CNN, LSTM-CNN, LSTM, and RF under various measures for two datasets. The hybrid model had the greatest accuracy of 0.95 at a learning percentage of 80% for the CK+ dataset. Also, the hybrid model has a higher sensitivity of 60% for FER-2013 datasets.