Layer-First “Cache-Rollback” Collaborative Pruning Method

With the significant enhancement of inference and storage capabilities of edge computing chips, inference speed has become the primary focus on the model compression. Existing network pruning methods, which is a classical model compression technique, face two major issues in real-time inference scenarios: firstly, a lack of differentiation in the priority of optimization objectives, leading to an overly large search space; secondly, while conventional pruning methods excel in parameter and computational compression, they do not exhibit great effects on inference acceleration. We propose a layer-first “cache-rollback” collaborative pruning method to solve them. Initially, we introduce a layer-first strategy. Subsequently, we design a layer importance evaluation method based on model recovery force, providing a reliable basis for efficient layer pruning. Finally, we design a “cache-rollback” mechanism, facilitating the efficient collaborative pruning of layers and channels, and validate its effectiveness in reducing the search space through theoretical analysis. Experimental results on mainstream edge inference chips reveal that: firstly, under the premise of within 10% decline in accuracy, our method achieves an average acceleration ratio of 2.36, an average parameter compression rate of 68%, and an average computational compression rate of 71%; secondly, compared with the currently fastest pruning algorithm, our method further increases the model acceleration ratio by 0.26 while improving the model accuracy rate by 0.47%.