Deviation-Sensitive Black-Box Anomaly Attribution

When the prediction of a black-box machine learning model deviates from the true observation, what can be inferred about the reason for that deviation? Such a deviation may arise from suboptimal model performance or the input being an outlier. In both scenarios, it is desirable to compute an attribution score that quantifies the contribution of each input variable to the anomaly. This task is known as anomaly attribution. In this chapter, we provide a comprehensive review of existing approaches to anomaly attribution in scenarios where neither the internal workings of the prediction model nor access to training data are available. Focusing on the regression setting, we first examine mainstream attribution methods, such as Shapley values, and highlight their unsuitability for anomaly attribution due to their deviation-agnostic nature. Motivated by this finding, we introduce a new paradigm called likelihood-based attribution and discuss two recently proposed methods: likelihood compensation (LC) and generative perturbation analysis (GPA). We highlight the functional advantages of these approaches while also discussing their limitations in comparison to existing alternatives.