Patient Data Analytics Using XAI: Existing Tools and Case Studies

Artificial intelligence (AI)-based systems have found extensive application within the healthcare sector. In healthcare, AI systems predominantly offer recommendations to physicians based on the analysis of patient health data. Typically, a doctor reviews diagnostic reports, assesses patient symptoms, and subsequently arrives at a diagnosis grounded in a comprehensible rationale. In contrast, when an AI system endeavors to emulate a doctor’s decision-making process, it can invite criticism due to its adherence to a black box approach when making critical determinations about patient well-being. This approach raises the potential for queries encompassing medical-legal, ethical, and societal dimensions concerning the guidance provided by the AI model. Consequently, there exists an imperative for the integration of explainable AI (XAI) within patient data analytics (PDA). XAI serves as an essential requirement within this context, unveiling the decision-making procedures previously veiled within the opaque construct of deep learning’s black box model. This chapter casts a spotlight on the pivotal role of XAI within medical systems. It accomplishes this by delving into the essence of XAI, elucidating its various categories, exploring the algorithms harnessed to unveil concealed information within black box systems, and addressing the challenges inherent to XAI. Furthermore, the chapter offers guidance to its readers on constructing intelligible deep learning models tailored for patient data analytics.