As Convolutional Neural Networks (CNNs) continue to scale in complexity, the concomitant demand for computational resources and storage capacity experiences a tremendous surge, which poses formidable challenges to deploying models within resource-constrained environments. Addressing this concern, channel pruning emerges as a significant method for mitigating the computational overhead and complexity of CNNs. This paper introduces a novel channel pruning method based on \(\boldsymbol{\ell _{0}}\) -norm sparse optimization, termed \(\boldsymbol{\ell _{0}}\) -norm Pruner. The \(\boldsymbol{\ell _{0}}\) -norm Pruner formulates channel pruning as a sparse optimization problem involving the \(\boldsymbol{\ell _{0}}\) -norm. Inspired by the problem-solving process, a Zero Norm (ZN) module is proposed. This module can autonomously select the output channels for each layer within the model according to a preset global channel pruning ratio. This method enables precise control over the pruning ratio, achieving effective pruning of complex models. Our experiments demonstrate that the proposed pruning method outperforms several state-of-the-art approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Explore Channel Pruning Based on  \(\ell _{0}\) -Norm Sparse Optimization

  • Enhao Chen,
  • Hao Wang,
  • Chi-Sing Leung,
  • Yuxin He

摘要

As Convolutional Neural Networks (CNNs) continue to scale in complexity, the concomitant demand for computational resources and storage capacity experiences a tremendous surge, which poses formidable challenges to deploying models within resource-constrained environments. Addressing this concern, channel pruning emerges as a significant method for mitigating the computational overhead and complexity of CNNs. This paper introduces a novel channel pruning method based on \(\boldsymbol{\ell _{0}}\) -norm sparse optimization, termed \(\boldsymbol{\ell _{0}}\) -norm Pruner. The \(\boldsymbol{\ell _{0}}\) -norm Pruner formulates channel pruning as a sparse optimization problem involving the \(\boldsymbol{\ell _{0}}\) -norm. Inspired by the problem-solving process, a Zero Norm (ZN) module is proposed. This module can autonomously select the output channels for each layer within the model according to a preset global channel pruning ratio. This method enables precise control over the pruning ratio, achieving effective pruning of complex models. Our experiments demonstrate that the proposed pruning method outperforms several state-of-the-art approaches.