Towards Better Evaluations of Class Activation Mapping and Interpretability of CNNs

As deep learning has been widely used in real life, there is an increasing demand for its transparency and its interpretability has received much attention from all walks of life. Current efforts in this field includes post-hoc visualization technique and intrinsically interpretable framework. However, there are still shortcomings in both of these techniques. In the post-hoc visualization techniques, the metric evaluating CAM method suffers from the ambiguity of evaluation object. We proposed a pair of quantitative evaluation metrics based on threshold cropping. The explanation maps were obtained by threshold cropping in order to minimize the variation of input images, thus making the evaluation object more focused on the CAM method itself. Experimental results show that these metrics can evaluate the accuracy and intensity of various CAM methods comprehensively, and get the conclusion that the gradient-based CAM methods are more stable, and Score-CAM is susceptible to model. Meanwhile, most of the existing intrinsically interpretable frameworks tend to enhance the interpretability of models by mapping their intermediate results to concepts that can be understood by humans. However, the intermediate results of the model often contain a variety of information that hinders its correspondence with single concept, which is manifested as a many-to-many relationship between filters and classes in convolutional classification networks. To address this situation, we proposed an interpretable training framework based on mutual information neural maximization to alleviate filter-class entanglement. MIS metric, classification confusion matrix and adversarial attack experiments all confirmed the validity of this method.