Computer Vision System for Monitoring User Attention in Interactive User Interfaces

The paper presents a new solution of computer-human interfaces personalization based on external analysis of the users’ activity based on face recognition. The approach includes tracking of the human face parts and emotion recognition using an external video camera integrated with an intelligent computer vision system. The knowledge base is used to compare the data from a computer vision system and action tracker and process it in a recognizer component. In case the identified activity (like emotion of number of actions) differs from the target one there are generated virtual stimuli that attract the user attention and provide the required activity change. The proposed approach was used in a medical system for psychological testing and also for proctoring in a digital educational platform. Testing experiments included 5000 images and proved a possibility to fairly clearly distinguish between attentive users and users being distracted. Both applications gathering an independent feedback form users in order to control the correct system functioning and adapt the components of user interface for user attraction when needed.