Explanatory systems (“explainers”) make the behavior of blackbox machine learning models more transparent. However, the results of different explainers (“explanations”) are often inconsistent with each other. To measure such differences, we propose a novel metric, the Shreyan Distance \(d_S\) . This paper uses \(d_S\) to compare two explanatory systems, SHAP and LIME, for both regression and classification learning tasks on tabular data. We find that the linearly weighted mean \(d_S\) value varies significantly between these two tasks. This leads us to conclude that consistency between explainers depends not only on inherent properties of the explainers themselves, but also on the type of learning task. In the hopes that more accurately quantifying differences between existing explainers will lead to better explanations in the future, we demonstrate a way to integrate our novel metric into existing machine learning pipelines via the XAISuite library.