Towards Engineering Explainable Autonomous Systems

Explanation is important to supporting appropriate levels of trust in autonomous systems. However, work in XAI (eXplainable AI) is focused on explanation of single system components, such as a machine learning algorithm or decision-making module. This paper: (1) argues that we need to develop ways to engineer explainable systems consisting of multiple components, and identifies this as a challenge for the community; (2) proposes an approach for explaining multi-component autonomous systems; (3) identifies integration issues that need to be addressed to make this vision a reality; and (4) poses a number of research challenges and questions that need to be addressed.