Evolutionary Weight Pruning: A PSO-Based Approach
摘要
Weight pruning is a widely adopted technique for compressing deep neural networks by reducing the number of parameters. However, most existing pruning approaches rely on removing individual weights based on magnitude thresholds, sparsity constraints, or heuristic rules, which often leads to suboptimal pruning outcomes and degraded model performance. In this paper, we propose a novel weight pruning method called PSOWeightPruner, which leverages the Particle Swarm Optimization (PSO) algorithm to automatically discover optimal pruning configurations. Rather than relying on manually defined pruning criteria, PSOWeightPruner formulates the pruning process as an optimization problem, where each particle in the swarm encodes a candidate pruning strategy. To efficiently explore the vast search space, PSOWeightPruner iteratively updates the particles’ positions and velocities, utilizing the global search capability of PSO to converge towards optimal solutions. This method eliminates the need for manual intervention and identifies superior pruning patterns that are often unattainable through traditional techniques. Extensive experiments show that PSOWeightPruner significantly outperforms conventional weight pruning methods in terms of both compression ratio and model accuracy, enabling efficient end-to-end fine-tuning with minimal performance loss.