Dealing with changes in data distributions is an important aspect of deploying machine learning based systems, including network intrusion detectors. While traditional drift detectors identify changes in feature distributions, they cannot reveal whether the decision boundaries of the model remain valid. This paper proposes leveraging Shapley Additive Explanation attributions to monitor shifts in the underlying feature-to-classification mapping. Using two NetFlow-based datasets (CIC-IDS2018 and UNSW-NB15), gradual multiplicative drift and an abrupt cross-dataset shift are simulated. For each drift scenario, raw feature distributions and Shapley Additive Explanations value distributions are compared via the Kolmogorov–Smirnov test and normalised Wasserstein distance. Results show that under pure scale drift, classification performance degrades and traditional detectors fire alarms; yet, the TreeSHAP explainability distributions remain stable. By differentiating drift types, the proposed explainability-based approach informs whether simple preprocessing adjustments suffice or if costly retraining is required.