Weight pruning is a widely adopted technique for compressing deep neural networks by reducing the number of parameters. However, most existing pruning approaches rely on removing individual weights based on magnitude thresholds, sparsity constraints, or heuristic rules, which often leads to suboptimal pruning outcomes and degraded model performance. In this paper, we propose a novel weight pruning method called PSOWeightPruner, which leverages the Particle Swarm Optimization (PSO) algorithm to automatically discover optimal pruning configurations. Rather than relying on manually defined pruning criteria, PSOWeightPruner formulates the pruning process as an optimization problem, where each particle in the swarm encodes a candidate pruning strategy. To efficiently explore the vast search space, PSOWeightPruner iteratively updates the particles’ positions and velocities, utilizing the global search capability of PSO to converge towards optimal solutions. This method eliminates the need for manual intervention and identifies superior pruning patterns that are often unattainable through traditional techniques. Extensive experiments show that PSOWeightPruner significantly outperforms conventional weight pruning methods in terms of both compression ratio and model accuracy, enabling efficient end-to-end fine-tuning with minimal performance loss.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evolutionary Weight Pruning: A PSO-Based Approach

  • Nian Wei,
  • Rongzuo Guo,
  • Jun Huang

摘要

Weight pruning is a widely adopted technique for compressing deep neural networks by reducing the number of parameters. However, most existing pruning approaches rely on removing individual weights based on magnitude thresholds, sparsity constraints, or heuristic rules, which often leads to suboptimal pruning outcomes and degraded model performance. In this paper, we propose a novel weight pruning method called PSOWeightPruner, which leverages the Particle Swarm Optimization (PSO) algorithm to automatically discover optimal pruning configurations. Rather than relying on manually defined pruning criteria, PSOWeightPruner formulates the pruning process as an optimization problem, where each particle in the swarm encodes a candidate pruning strategy. To efficiently explore the vast search space, PSOWeightPruner iteratively updates the particles’ positions and velocities, utilizing the global search capability of PSO to converge towards optimal solutions. This method eliminates the need for manual intervention and identifies superior pruning patterns that are often unattainable through traditional techniques. Extensive experiments show that PSOWeightPruner significantly outperforms conventional weight pruning methods in terms of both compression ratio and model accuracy, enabling efficient end-to-end fine-tuning with minimal performance loss.