Exceptional Model Mining (EMM) seeks subgroups of the dataset that are interpretable, and display exceptional behavior. We make contributions to both of these EMM properties. To behavior, we contribute Exceptional Model Residual Mining (EMRM), gauging exceptionality of subgroup behavior on the residuals of a regression model. Existing versions of EMM with regression models focus on exceptionality of regression parameters themselves; EMRM instead discovers subgroups explaining systematic model under- or overperformance. To interpretability, we contribute three description languages. Existing work almost always restricts the subgroup search to those subsets that belong to the Conjunctive description language; we propose Polynomial Predicates, Shallow Decision Trees, and Symbolic Expressions as alternatives. Experiments on five UCI datasets illustrate what kind of subgroups EMRM can find, when optimizing for high (underperformance) and low (overperformance) residuals. We also compare the four languages, by inspecting how they fare in trading off subgroup quality and interpretability: Symbolic Expressions achieve the best trade-off, capturing highly exceptional regions with minimal complexity, while Conjunctions achieve the highest peak quality across the majority of datasets.