Deep Feature Discriminability as a Diagnostic Measure of Overfitting in CNN Models

In computer vision, a deep understanding of overfitting phenomena in Convolutional Neural Networks(CNNs) plays a pivotal role in developing novel, advanced Deep Learning architectures. In this study, we present a novel methodology that identifies and analyzes model overfitting by leveraging unsupervised clustering of the features extracted by CNNs. Our research demonstrates that overfitted models exhibit inadequate class discriminability within the feature space. To further delve into the distribution of classes in the deep feature space, our study combines conventional cluster separability metrics with model selection methods, resulting in a comprehensive characterization of the training process and the resulting trained architecture. We show that poor training procedures, such as overfitting or insufficient training data, directly compromise the discriminative capability of deep features in terms of two novel evaluation metrics: cluster-discriminability and cluster-class similarity scores. We also investigate the impact of transfer learning and dropout regularization on overfitting. Experimental results on benchmark datasets for classification tasks validate that transfer learning is less susceptible to overfitting, and incorporating dropout regularization effectively mitigates overfitting tendencies and enhances the generalization capabilities of CNNs. Furthermore, we unveil that the cluster-discriminability score can serve as a metric to distinguish between normal and adversarial inputs, making it a valuable tool for detecting adversarial attacks in CNNs.