Network Pruning via Explicit Information Migration

Neural network pruning is a widely used approach for reducing the inference cost of deep models in order to deploy on resource-limited settings. However, current pruning works lack attention to information migration from pruned to the remaining part of the Deep Neural Network, and on balancing model performance and compression rate. On these two issues, in this paper, we propose a novel \(\textbf{E}\) xplicit \(\textbf{I}\) nformation \(\textbf{M}\) igration network \(\textbf{P}\) runing (EIMP) algorithm. Specifically (1) the constrained gradient update method transfers valid information from redundant networks to the preserved, and (2) the newly designed \(\lambda \) -decay regularization method learns the trade-off between the performance and penalty item. Experiments show that our EIMP algorithm achieves state-of-the-art performance on several datasets with various benchmark network architectures. Notably, EIMP achieves \(+1.54\%\) better than SOTA on ImageNet.