In this study, we center our focus on how interpretability works in the presence and absence of defence mechanisms for adversarial attacks. We scrutinise the susceptibility of medical image classification models to adversarial attacks, specifically honing in on the Adversarial Imperceptible Patch Attack. By employing the GradientSHAP method, we delve into the vulnerabilities of the VGG16 model trained on SARS-CoV-2 CT-scan and Malaria Cell Image datasets, seeking to understand how it responds to adversarial perturbations. Additionally, we explore defensive mechanisms such as adversarial training and denoise filtering to mitigate the impact of these attacks. We study how interpretability using GradientSHAP varies after the attack and how it varies again after the defence mechanisms are employed. Our work deepens stakeholder understanding of AI diagnostic risks, strengthens model resilience, and promotes confidence in healthcare integration.