Decision Fusion-Based System for Automatic Face Emotions Recognition

Facial expressions play an important role in communication and interpersonal relationships. That is why a variety of applications have been developed in various fields in which they must be recognized, such as security, medicine, psychology, marketing, education, gaming, etc. There are generally six basic emotions that can be identified from facial images and a few derived emotions. Thus, the paper proposes a system based on the collective intelligence of five neural networks (MobileNetV2, MobileNetV3 Large, EfficientNet B0, EfficientNet B2, and EfficientNet B5) for the recognition of emotions with the goal of early detection of mental illness and rapid introduction of a therapeutic scheme. The decision fusion method considers the maximum sum of the individual predictions of the 5 networks per class and the validation of the resulting class by at least two individual networks. The testing of the global system was done on three emotions, sad, angry, and happy using images from the public database, and the accuracy obtained was 84.73%, better than the individual neural networks. The results obtained are promising and lead to the use of this system and other categories of facial expressions.