An Interpretability Case Study of Unknown Unknowns Taking Clothes Image Classification CNNs as an Example

“Unknown unknowns” are instances predicted models assign incorrect labels with high confidence, greatly reducing the generalization ability of models. In practical applications, unknown unknowns may lead to significant decision-making mistakes and reduce the application value of models. As unknown unknowns are agnostic to models, it is extremely difficult to figure out why models would make highly confident but incorrect predictions. In this paper, based on identification of unknown unknowns, we investigate the interpretability of unknown unknowns arising from convolutional neural network models in image classification tasks by interpretable methods. We employ visualization methods to interpret prediction results on unknown unknowns, further understand predictive models and analyze the predictive basis of unknown unknowns. We focus the application scenario of interpretability of unknown unknowns on a clothes category recognition task (dress vs shorts) in e-commerce platforms, and observe some patterns of models making wrong classifications that lead to unknown unknowns, which indicates that a CNN model that lacks of common sense can make mistakes even for a large dataset. Besides, we observe some interesting phenomena: certain correct predictions of instances are unreliable due to wrongly identified features by CNNs.