Real-Time Student Behavior Recognition in Classroom Using Pose Estimation and Gaze Analysis
摘要
In current education scenarios, the monitoring of attention levels among students constitutes an important aspect in increasing teaching effectiveness. In this paper, we bring forth a system for recognizing classroom behavior based on visible cues that employs both gaze direction detection and posture estimation. System architecture constitutes a combination of three primary modules, namely Faster R-CNN for student recognition, HRNet for extracting posture, and L2CS-Net for estimating gaze. Features obtained from the HRNet are embedded in a convolutional neural network (CNN) that places student activity in four respective classes: looking, asking, boring, and bowing. Experimental evaluations utilizing a real-world testing set of 832 images establish the system as having remarkable classification capacity, obtaining an average F1-score of 0.9925. Notably, inquiring and boring obtained perfect precision and recall values (1.0), which establish superior recognition mastery. Bending behavior achieved a recall of 0.98 and an F1-score of 0.99, and the stare feature set obtained equivalent values, including a precision value of 0.98 and a recall value of 0.99. Utilization of pre-trained deep learning models efficiently mitigates the workload of data annotation costs while, in addition, improving functional practicability. Incorporating this approach constitutes a prospective measure toward implementation in smart classroom technologies, allowing teachers to perceive student behavior in a salient, accurate, and real-time format.