Towards the Design of Explanation-aware Decision Support Systems

Explanation capability is crucial to ensure responsible and trustworthy Artificial Intelligence (AI). The need for human-centric explanation is critical to increasing the uptake of explainable AI (XAI) systems. This paper emphasises the need to integrate stakeholder factors into the development of explainable decision support systems. Much research efforts have focused on explainability, but the integration of stakeholder factors into AI-generated explanations is still lacking. To demonstrate our notion of explanation-awareness of decision support systems, we used the example of the design of an explanation-aware academic advising system. We identified the different stakeholders of the system and showed how stakeholder factors such as identity, preferences, and characteristics can be integrated into the design of the system. We also identified relevant metrics that can be used to evaluate explanation-aware decision systems. As a contribution, this paper extends to the ongoing discussion on the design and development of trustworthy and transparent explainable decision support systems, which are needed but are not yet common.