This work shows an extensive multimodal system of mood detection and customized playlist recommendation based on EEG, GSR, and face recognition. Brainwave activity for emotional evaluation is sensed by EEG electrodes, while GSR sensors provide skin conductance, heart rate variability, and temperature values for physiological behavior. Facial behavior with emotional facial expressions is determined through facial landmarks. Preprocessing entails Butterworth filters for EEG frequency bands, GSR data normalization, and facial feature extraction from a pretrained model (FER) to track eyebrow position, mouth curvature, and eye openness. EEG features are examined using frequency domain analysis, whereas GSR and facial features are classified using Random Forest. To increase precision, a fusion model aggregates predictions by weighted averaging or majority voting, with EEG assigned the greatest weight due to its high correlation with mood. After determining the emotional state, a suitable playlist is suggested: energetic songs for happiness, relaxing music for stress, comforting songs for sadness, and relaxing music for relaxation. This feature-based recommendation system enhances personalization through the use of features like tempo, genre, and mood to provide a dynamic and interactive listening experience for the user.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Playlist Recommendation System

  • M. S. Shalini,
  • B. A. Sujathakumari,
  • Lalithya Govardhan,
  • P. S. Gagan Deep

摘要

This work shows an extensive multimodal system of mood detection and customized playlist recommendation based on EEG, GSR, and face recognition. Brainwave activity for emotional evaluation is sensed by EEG electrodes, while GSR sensors provide skin conductance, heart rate variability, and temperature values for physiological behavior. Facial behavior with emotional facial expressions is determined through facial landmarks. Preprocessing entails Butterworth filters for EEG frequency bands, GSR data normalization, and facial feature extraction from a pretrained model (FER) to track eyebrow position, mouth curvature, and eye openness. EEG features are examined using frequency domain analysis, whereas GSR and facial features are classified using Random Forest. To increase precision, a fusion model aggregates predictions by weighted averaging or majority voting, with EEG assigned the greatest weight due to its high correlation with mood. After determining the emotional state, a suitable playlist is suggested: energetic songs for happiness, relaxing music for stress, comforting songs for sadness, and relaxing music for relaxation. This feature-based recommendation system enhances personalization through the use of features like tempo, genre, and mood to provide a dynamic and interactive listening experience for the user.