Skeleton-Based Student Activities Recognition from Classroom Videos

Human activity recognition has become a popular area of research in recent years due to its practical applications. While there have been many studies on activity recognition in general, identifying the activities of students in a classroom presents unique challenges. This is because multiple individuals are engaging in different activities at different times. Additionally, there is a limited amount of data available on students’ activities for training and benchmarking recognition models. To address these challenges, we collected a dataset that includes the main activities of students in classrooms, such as sitting and studying, raising hands, standing up, using phones, and sleeping. We then developed a recognition method that extracts human skeletons from RGB images using AlphaPose and recognizes activities from the temporal sequence of the extracted skeletons with the DD-net model. This approach avoids the issue of similar color and texture in students’ clothing by using skeleton modality. Furthermore, temporal-spatial features provide better discrimination of activities than spatial features at the frame level. Our proposed method was evaluated on a self-collected dataset of five students’ activities in real classrooms and showed promising results. It could serve as a useful reference for benchmarking student activity recognition models.