Music Recommendation Based on Face Emotion Recognition

The paper presents a mobile application, EmoTunes, that detects the face emotions of people and thereafter plays a suitable music. The emotion detection system works for any skin tone as the dataset introduced contains a mix of ethnicities. The dataset was collected by asking friends, family members and acquaintances selfies of themselves expressing the Ekman’s basic emotions. EmoTunes then used an ensemble learning with MobileNetV2 and ResNet50 to detect the 7 Ekman’s emotions: Angry, Disgust, Fear, Happy, Neutral, Sad and Surprise. The system can detect single user as well as multi-user emotions. It has 2 types of recommendation: Recommendation from server, where it fetches songs from the database, and Recommendation via YouTube, where it takes only a single emotion and fetches a YouTube song that best relates to the user’s preferences and liking. EmoTunes uses 2 methods of discretizing multi-user emotions: either by finding the most common emotion among them, or by finding the emotion of the face nearest to the camera. During real-time emotion detection, the system detects the user’s emotion for 15 s and finds the most common emotion expressed within those 15 s to play a music directly from the server. A total validation accuracy of 99.64% was deduced from the final model, making it one of the promising models when compared to recent research work. The system can be deployed to any part of the world as it adapts to each user’s liking using the YouTube feature. While humans are social creatures, they do not like to wait for services. EmoTunes can hence be deployed in waiting rooms such as for healthcare facilities and at the metro/bus stations, or even in buses and trains in Mauritius, or other parts of the world, to provide the users with some recomforting music based on their current emotions. Future considerations might include further working on the dataset to achieve a much bigger variation.