Using Explainability to Find Spurious Patterns in Textual Datasets

Despite advances in AI, many algorithms are still considered “black boxes” due to a lack of transparency and interpretability. This opacity generates insecurity in users, who want to understand how the models arrive at certain decisions and whether the patterns found are genuinely relevant. Spurious patterns, apparent correlations between attributes and classes with no causal relationship, can lead to inaccurate results, bias, and difficulties in interpreting AI models. This article aims to present an end-to-end pipeline for detecting and mitigating spurious patterns in textual datasets using Explainable AI (XAI) techniques. Distinct from other approaches, this work uses global and local weights given by explainers to uncover spurious patterns and uniquely identifies these patterns without prior knowledge of the test data. While showing modest improvements in Accuracy and F1-Score metrics, the pipeline demonstrated a capability to identify spurious patterns, detect labeling errors, and enhance the interpretability of decisions made by the Explainer Model. These advancements improved dataset quality and model accuracy, indicating that the pipeline can be effectively utilized in practical scenarios, yielding tangible benefits and bolstering the reliability of classifications.