Research on the Influence of Activation Functions on the Performance of Convolutional Neural Networks and the Effect of Channel Pruning

To address the issues of high computational cost and memory requirements when deploying deep convolutional neural networks on resource—constrained devices, channel pruning is an effective model compression method. However, existing research has insufficient attention to the role of activation functions in pruning. This paper takes VGG16 as the research object, explores the impact of activation functions on channel pruning, proposes a new adaptive activation function AdaLReLU, and compares it with ReLU, LeakyReLU, sigmoid, and tanh. The activation function is uniformly replaced during the model training and the fine—tuning stage after pruning. A pruning algorithm based on cosine similarity DBSCAN clustering is adopted, and the effects are evaluated through classification accuracy, parameter reduction rate, and FLOPs reduction rate. The results show that the activation function has a significant impact on the performance after pruning. AdaLReLU solves the problems of poor adaptability of fixed parameters and gradient propagation of traditional activation functions by dynamically adjusting the slope and introducing learnable parameters. The test accuracy after pruning reaches 93.32% (only a 0.69% decrease), while achieving a parameter reduction rate of 93.88% and a FLOPs reduction rate of 69.22%. It has the best balance between accuracy retention and compression efficiency. Compared with methods such as SAFS and PReLU, it is superior in terms of baseline accuracy and accuracy stability before and after pruning, confirming the key role of activation functions in model compression and providing new ideas for the collaborative design of activation functions and pruning algorithms.