Analyzing Students’ Emotion and Activities in the Classroom: A Rural Education Perspective

Analyzing the concentration levels of students in a classroom and locating their positions pose significant challenges due to varying environmental conditions and diverse features. Facial expressions can reflect an individual’s mood, with different muscle positions beneath the skin indicating various emotions. Head orientation serves as an additional parameter to generalize student concentration levels. Moreover, different body postures can also provide insights into a person’s mood and concentration. This paper presents a lightweight application designed to analyze student concentration levels in a classroom, particularly to support rural education systems. Our work aims to address the challenges of cost-effectiveness and lightweight implementation suitable for running on affordable smartphones commonly used by rural faculty members. We employ the popular Dlib library for face detection, the FaceNet model for feature extraction, and the Siamese network for face recognition. We achieve 95% accuracy in face recognition using publicly available celebrity datasets, and our in-house dataset collected from NIT, Durgapur yields an 88% accuracy. Additionally, we implement MobileNet transfer learning for emotion recognition, focusing on relevant emotions such as Anger, Disgust, Fear, Happy, Neutral, Sad, and Surprise, achieving an accuracy of 73.76%. For Human Activity Recognition, we consider activities such as Clapping, Reading, Raising Hand, Looking Around, and Standing. We employ R-CNN and Random Forest techniques, achieving an accuracy of 64.92% on the in-house NIT, Durgapur dataset. Finally, we develop a mobile app that encompasses all functionalities, including class recording, data synchronization with Internet storage, and registration features for students and volunteers.