Machine learning (ML) and deep learning (DL)-based Intrusion Detection Systems (IDS) have shown promise but remain highly vulnerable to adversarial examples (AEs) – specially crafted inputs designed to evade detection – posing serious security risks. Moreover, their black-box nature limits explainability, undermining trust and hindering defense development. To address these challenges, we propose X-AdvIDS, a novel framework combining adversarial robustness and IDS explainability. Specifically, X-AdvIDS consists of two key modules: Adv-Sword, which leverages explainable Artificial Intelligence (XAI) to generate high-evasion AEs for assessing IDS weaknesses, and Adv-Shield, which utilizes explainable AI to construct a whitelist of trusted features for adversarial sample detection. Experiments on the InSDN and CICIDS2018 demonstrate that Adv-Sword significantly reduces IDS detection performance, revealing vulnerabilities, while Adv-Shield detects over 90% of adversarial inputs with low false positives. Compared to existing methods, X-AdvIDS enhances both robustness and interpretability, making IDS more resilient and transparent.