A Filter Similarity-Based Early Pruning Methods for Compressing CNNs

There are a large number of redundant features in deep neural networks, the pruning method based on redundant filters can identify redundant features and prune them. However, these methods require a long-term pre-training, which greatly increases the computational cost. Inspired by the early-bird tickets, this paper proposes an early pruning algorithm based on filter similarity. First, the C-SGD optimizer is used in training to move the filter towards the cluster center. At the same time, we propose an Intra-Cluster Similarity Metric to measure the degree of redundancy in a cluster, when the similarity reaches the specified threshold, it means that the filters in the cluster are redundant and the network is stable, and then the redundant filters can be safely removed. Our method can greatly re-duce the training time and compress the network, and the pruned model can achieve the accuracy of the original model after retraining. Our algorithm pruned ResNet-56 and was able to reduce 60.8%FLOPs on CIFAR-10 with 0.16% accuracy degradation. For ResNet-50, it can reduce FLOPs by 55.7%, TOP1 accuracy is only reduced by 1.37%, and the time required for training can be greatly reduced.