Through the Eyes of the Expert: Aligning Human and Machine Attention for Industrial AI

Human expertise and intuition are crucial in solving many tasks in expert-driven domains such as industrial manufacturing or medical diagnosis. In this work, we use the human expert’s gaze information to take a step towards transferring this knowledge to a machine learning model. In this way, we are aligning the attention the machine and the human pay to solve the task. Previous works in the medical field have shown that privileged gaze information during training can increase predictive performance and reduce the label requirement of a machine learning model. We extend on the aim of those works and quantitatively evaluate the benefit of aligning human and machine attention on the quality of the model’s explanations as well as its robustness - thus, its trustworthiness. We demonstrate our approach on a real-world visual quality inspection task in the multi-label setting, which is common in industrial applications. Our work illustrates the importance of incorporating human knowledge more explicitly in training machine learning models and takes a step towards enabling machine learning based systems in high-stakes applications.