A review on speech emotion recognition for late deafened educators in online education

In an online class, students respond to queries initiated by the educator through voice, video, or chat. The educator receives these responses but cannot ascertain the emotion carried in these responses with confidence. This emotional limitation in online classes is a major drawback as emotions play a very important role in gauging student engagement and learning. This limitation in the detection of emotions or affective states in the responses of the student makes it difficult for the educator to decide if the student is on the right track or needs more help. On the other hand, if a negative affective state is detected, the educator could modify their teaching approach, speed, or content to the advantage of the student. This poses as an additional disadvantage to educators who suffer from any form of hearing impairment like late or acquired deafness, as they would have difficulty in hearing as well as gauging the emotion from student’s feedback. Emotions being a key element in decision making and speech being one of the affective measurement channels, a useful solution would be to use an automatic speech emotion recognition system. In this paper, speech emotion recognition systems (SER) have been reviewed with the intention of identifying research gaps that need to be bridged before an effective SER application can be incorporated in any online education system for teaching. This would guarantee inclusivity as it would benefit all educators, especially the late deafened. This paper has been written with the intention of providing an in-depth understanding of SER components, spanning from emotional models to classifiers. The opportunities for possible improvements have also been touched upon. Special attention is given to deep learning approaches to understand their main contributions and challenges so far in this area.