Student Behavior Analysis in Classroom Using Facial Emotion Recognitions and Graph Visualizations

Majority of the present generation of people is felt hard to handle their emotions due to many reasons. There is a drastic change in the behavior of students in the classroom environment which reflects in the emotional quotient (EQ). Few researchers contributed on emotions of learners. The novice faculty needs lot of skills to engage the class either online or offline mode and to ensure the learning happens effectively. There is a need for an automatic emotion recognition system in the classroom to facilitate the faculty. In this work, an automatic facial emotion recognition system is proposed to measure the students’ facial expressions such as listening, joy, drowsy, and confusion during the class. The analysis dashboard of the proposed system indicates the student engagement index (SEI) that helps the faculty to improve their teaching plan and to enhance their class handling skill. The Viola–Jones algorithm is used along with the convolutional neural network (CNN) model to implement the proposed system. The model was trained using FER 2013 and MobileNetV2 datasets for facial features detection and for emotions recognitions. The proposed system is working with 88% accuracy. The change of emotional states of each student for each class is visualized graphically. Hence, it is proved that the proposed system will be highly suitable for both online and offline classes.