Does the Model Say What the Data Says? A Simple Heuristic for Model–Data Alignment
摘要
In this work, we propose a simple, computationally efficient framework to evaluate whether machine learning models align with the structure of the data they learn from, that is, whether the model says what the data says. Unlike existing interpretability methods that focus exclusively on explaining model behavior, our approach establishes a baseline derived directly from the data itself. Drawing inspiration from Rubin’s Potential Outcomes Framework, we measure how strongly each feature separates two outcome groups in a binary classification task, going beyond traditional descriptive statistics to quantify each feature’s effect on the outcome. By comparing these data-derived feature rankings against model-based explanations, we provide practitioners with an interpretable method to assess model–data alignment.