Assessing Student Engagement Levels Using Speech Emotion Recognition

Almost all instructors face the challenge of understanding how engaged the participants or students are with the class sessions. Understanding the engagement levels of students is vital to grasp early warnings for any potential dropouts. This study proposes using Speech Emotion Recognition to understand the engagement levels of students by grasping the emotions underneath the verbal feedback given by students for the conducted class sessions. Instead of focusing on the semantic contents of the speech, Speech Emotion Recognition focuses on the emotion(s) contained in the speech signal. This work describes the development stages of a Speech Emotion Recognition Model to deduce the student’s emotional state while giving verbal feedback on conducted class sessions. This study considered various approaches based on machine learning classification techniques, such as support vector machines, random forest, decision tree, extreme gradient boosting, and deep learning classification techniques based on long short-term memory. Various features from the speech signals were extracted and narrowed down to features of importance for the classification task. The developed Speech Emotion Recognition Models are trained and tested on the Ryerson Audio-Visual Database of Emotional Speech and Song and the Toronto Emotional Speech Set. The respective accuracy values of the developed models are 84.27% for the machine learning model and 84% for the deep neural network model. Instructors can use the developed system to assess student engagement levels to get early warning signals of potential dropouts.