Effective and understandable Intrusion Detection Systems (IDS) are now a necessity in the ever-evolving world of cybersecurity. Conventional IDS models, particularly deep learning models, at times are like “black boxes,” making it hard for security professionals to understand and trust their decisions. Explainable AI (XAI) offers a solution by enhancing transparency, allowing for effective threat avoidance methods, and providing explanations about model predictions. The central goal of this research is to evaluate the viability of two renowned XAI approaches in the application of IDS: SHapley Additive Explanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME). We compare these models based on their ability to elucidate the decision-making process of a deep learning based IDS from the CICIDS-2017 dataset. By closing the gap between interpretability and high-performance IDS, our work aims to shed light on how explainability can be leveraged to improve real-world cybersecurity operations.