Layer-Wise Pruning Ratios Auto-configuration: A One-Shot Channel Pruning Through Sensitivity and Spatial Analysis
摘要
Pruning is a common compression approach for neural networks. Existing structured pruning methods suffer two major issues: (1) coupling of analyzing pruning ratios configuration and which specific convolution kernels to remove from a certain layer, i.e., intra-layer pruning strategy, makes it difficult to evaluate their real impact, (2) insufficient consideration of spatial context results in the loss of target features, leading to a significant accuracy decline. To this end, we propose a Layer-wise Pruning Ratios Auto-configuration (LPRA) framework, offering a standardized pruning ratios configuration approach for various intra-layer pruning strategies, making it easy to compare their actual pruning effects. We simultaneously assess two metrics: Spatial Coherence (SC), representing the extraction capability of spatial features, and Hessian Mean (HM), representing the sensitivity of convolutional kernels, indicating the impact of the layer on the output. LPRA can preserve spatial information in the model while reduce its impact on accuracy, achieving efficient one-shot channel pruning without introducing any additional network modules or regularization losses. Experimental results show that LPRA performs well in both classification and segmentation tasks, pruning almost 50 \(\%\) of parameters from VGG-16 with 0.36 \(\%\) accuracy gain on CIFAR-100, pruning 41 \(\%\) of parameters from ResNet-34 with 0.16 \(\%\) accuracy gain on CIFAR-10. Furthermore, we identify redundancy in real-time semantic segmentation models.