Automated Facial Expression Analysis for Cognitive State Prediction During an Interaction with a Digital Interface

Automated multimodal facial expression analysis is an advanced technology used in user experience research to predict cognitive states during UX evaluation. It involves analyzing user is facial expressions as they interact with digital interfaces using various tools and techniques. The information obtained through this analysis can help evaluate the user experience, thereby leading to the development of more efficient and higher-quality digital products. The multimodal extraction strategy involves detecting 46 points related to head movement, hand position, and facial expressions. Three classification algorithms were analyzed in conjunction with the Cam3D and Pandora data sets. The results indicate that Random Forest achieved an accuracy of 98%, KNN achieved an accuracy of 97%, and SVM achieved an accuracy of 95% for the detection of attention, concentration, and distraction. Incorporating cognitive state detection during UX assessment represents a valuable opportunity to improve the quality and efficiency of digital products.