Channel pruning is a promising technique for compressing and accelerating deep learning models, but traditional methods require original training data and extensive fine-tuning, which can be impractical. In this paper, we present a novel accelerated Shapley based pruning method with data synthesis and channel dependency grouping, which eliminates the need for training data. Specifically, our approach first synthesizes data based on the statistical information in batch normalization layers of pre-trained networks. Furthermore, we introduce a layer-wise differential learning (LDL) module to enhance synthetic data diversity. Besides, an automated channel dependency grouping strategy is proposed to identify and group coupled channels, ensuring efficient and structured pruning while maintaining network connectivity. In addition, we leverage accelerated Shapley values to measure channel importance in an interpretable manner, reducing information loss during pruning. Extensive experiments of pruning VGGNet and ResNet on CIFAR-10, CIFAR-100 and ImageNet demonstrate that our method achieves superior performance in compression, acceleration, and accuracy compared to existing data-free pruning methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Accelerated Shapley Based Pruning with Data Synthesis and Channel Dependency Grouping

  • Zexu Huang,
  • Yihao Chen

摘要

Channel pruning is a promising technique for compressing and accelerating deep learning models, but traditional methods require original training data and extensive fine-tuning, which can be impractical. In this paper, we present a novel accelerated Shapley based pruning method with data synthesis and channel dependency grouping, which eliminates the need for training data. Specifically, our approach first synthesizes data based on the statistical information in batch normalization layers of pre-trained networks. Furthermore, we introduce a layer-wise differential learning (LDL) module to enhance synthetic data diversity. Besides, an automated channel dependency grouping strategy is proposed to identify and group coupled channels, ensuring efficient and structured pruning while maintaining network connectivity. In addition, we leverage accelerated Shapley values to measure channel importance in an interpretable manner, reducing information loss during pruning. Extensive experiments of pruning VGGNet and ResNet on CIFAR-10, CIFAR-100 and ImageNet demonstrate that our method achieves superior performance in compression, acceleration, and accuracy compared to existing data-free pruning methods.