Towards Quantification of Eye Contacts Between Trainee Doctors and Simulated Patients in Consultation Videos

Eye contact plays a critical mediating role in good non-verbal communication within the bigger umbrella of healthcare consultations, such as non-verbal communication/interactions between doctors and patients as it enhances patients’ satisfaction, trust and clinical outcomes. Assessing and coaching (training) trainees’ and practitioners’ communication skills is highly labour intensive and prone to subjectivity in medical education research. Moreover, the resulting judgements often lack specificity to really guide developments in performance and often involves subjective and social biases. In this work, we devise two plausible annotation schemes, developed deep convolutional neural network (DCNN) based architecture to categorise eye contact between two persons’ among medical consultation videos. We address the challenges faced and produce an authentic unconstrained training/testing dataset from real student exams: unconstrained and unobtrusive consultation between trainee doctors and simulated patients in medical education research. Our research on the challenging problem of unconstrained eye contacts quantification shows that the performance testing accuracy is promising, especially when DCNN based algorithms are used. It also provides useful insights on the evolving strategies of data annotations, biases involved, analysis and calls for development of more sophisticated algorithms for unconstrained eye contacts environment.