Twin attention based multi-task convolutional bidirectional long short term memory for facial expression recognition

Facial Expression Recognition (FER) aims to detect the emotional state of facial images. It is playing an increasingly important role in several application areas, including human–computer interaction (HCI), video transcriptions, and social communications. This article provides an adequate attention-based multi-task deep learning method for facial expression recognition. First, the input videos are collected from the RAVDESS and MELD datasets. Then, the input videos are converted using a threshold-based keyframe extraction algorithm. Next, the input data is pre-processed using the Adaptive Pixel Density Median Filtering (A-PDMF) method. Key features such as shape, color and texture are extracted from the pre-processed images. Finally, the facial expressions are recognized by proposing a novel twin attention-based multi-task convolutional bidirectional long-short-term memory method (TA-MC-BiLSTM). In addition, the classification parameters are optimally tuned using the EX-AHA method (Extended Artificial Hummingbird Algorithm). The proposed model reduces the size of facial features while accurately identifying the wide range of facial expressions. For simulation, the proposed method prefers python tool and results are analyzed using RAVDESS and MELD datasets. The simulation results show that the proposed model provides better results than other existing models in terms of accuracy of 99.6% for RAVEDESS and 99.4% for MELD.