The children’s psychological emotion recognition on STEAM education

Emotion recognition, a burgeoning field in recent years, involves computers discerning and comprehending human emotions, finding widespread applications. While facial expressions are commonly used for emotion recognition, the significance of body posture in this process often goes unnoticed. This study proposes a method to enhance emotion recognition accuracy in educational settings by combining facial expressions and body posture, grounded in the Science, Technology, Engineering, Art, Mathematics (STEAM) education paradigm. Initially, the relationship between STEAM education and children’s emotional regulation is examined to provide a theoretical underpinning. Subsequently, video data of children engaged in STEAM activities is briefly preprocessed, and emotional features are extracted using Visual Geometry Group Network (VGG)16 for facial expressions and movement-based analysis for body posture. The two modalities are then fused and classified, and the recognition accuracy of single and fused modes is compared. Results indicate that the average accuracy rates for facial expression and body posture recognition are 84.627% and 89.523%, respectively, while the combined mode achieves 94.135%. This method demonstrates robust generalization and applicability, emphasizing the complementary nature of emotional information conveyed through facial expressions and posture. Furthermore, automatic feature extraction using deep learning (DL) for posture emotion recognition shows promising results, suggesting the potential for DL to enhance static or dynamic posture representations. The integration of both recognition modes significantly improves accuracy, offering valuable insights for application and further research.