OpenFace Tracker and GoogleNet: To Track and Detect Emotional States for People with Asperger Syndrome

The goal of this work is to create an emotional model that can categorize emotions in real-time for persons with Asperger’s syndrome (AS). The model is based on facial expressions, head movement, and eye gaze as significant features for emotions. Developing like this an emotional model can aid other people to communicate socially with people with autism. This model overcomes the limitations of previous research that used sensitive and invasive methods to capture physiological data in order to predict emotional states. These instruments are very costly and need a controlled environment. The proposed model succeeded in classifying the emotional states of the AS using a natural spontaneous dataset without invasive tools and in an uncontrolled environment. The dataset used in this study is an available dataset and contains videos recorder in an uncontrolled environment with different facial occlusion and illumination changes. The emotions were classified as fear, disgust, joy, anticipation, and sadness. The proposed model implements a modified version of the well-known and wide-used GoogleNet machine learning model to classify emotions. The key feature of GoogleNet is the inception module, which is designed to capture different levels of spatial features and perform dimensionality reduction using parallel convolutional layers with different filter sizes. There are several metrics that were used to measure the accuracy and generality of a DL model, including accuracy, precision, recall, and F1 score, depending on the nature of the task. Overall, measuring the generality of a DL model is an important step in evaluating its performance and ensuring that it will perform well on new, unseen data. The model achieved significant performance on unseen data with accuracy (98%).