Scholarlink

<p>Large language models (LLMs) have become foundational in modern natural language processing, enabling diverse applications such as conversational AI, code generation, and scientific discovery. Their massive parameter counts impose significant computational and memory demands, hindering deployment in resource-constrained environments. Network pruning has become a practical solution used to compress LLMs by removing their redundant parameters. However, existing LLM pruning methods are primarily designed based on either structured or unstructured pruning, typically applying a uniform pruning strategy across all layers, and they fail to account for the heterogeneous sensitivity of different layers to sparsification. In real-world deployment scenarios such as edge-device inference, there is a pressing need for pruning strategies that dynamically adjust sparsity strategies across layers to satisfy diverse requirements on latency, accuracy, and energy efficiency. In this work, we propose Layer-wise <b>H</b>eterogeneity-guided <b>H</b>eterogeneous <b>P</b>runing (HHP), a method that adaptively combines structured and unstructured pruning across layers guided by each layer’s heterogeneity to compress LLMs while balancing inference performance and efficiency. HHP consists of two key components: (1) a layer-wise heterogeneity importance score that quantifies each layer’s sensitivity to different pruning granularities, and (2) an adaptive heterogeneous pruning method that uses this score select the appropriate pruning method for each layer. Experiments on LLaMA and other LLMs show that, at the 40% sparsity ratio, our method provides a controllable trade-off between performance and efficiency for pruned models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Layer-wise heterogeneity-guided heterogeneous pruning: An LLM compression method

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月30日

查看全文
  • Zhihao Yu,
  • Zuxin Ma,
  • Guowei Shen,
  • Yi Chen,
  • Chun Guo,
  • Xiaoqi Duan,
  • Qing Qian,
  • Yunhe Cui

摘要

Large language models (LLMs) have become foundational in modern natural language processing, enabling diverse applications such as conversational AI, code generation, and scientific discovery. Their massive parameter counts impose significant computational and memory demands, hindering deployment in resource-constrained environments. Network pruning has become a practical solution used to compress LLMs by removing their redundant parameters. However, existing LLM pruning methods are primarily designed based on either structured or unstructured pruning, typically applying a uniform pruning strategy across all layers, and they fail to account for the heterogeneous sensitivity of different layers to sparsification. In real-world deployment scenarios such as edge-device inference, there is a pressing need for pruning strategies that dynamically adjust sparsity strategies across layers to satisfy diverse requirements on latency, accuracy, and energy efficiency. In this work, we propose Layer-wise Heterogeneity-guided Heterogeneous Pruning (HHP), a method that adaptively combines structured and unstructured pruning across layers guided by each layer’s heterogeneity to compress LLMs while balancing inference performance and efficiency. HHP consists of two key components: (1) a layer-wise heterogeneity importance score that quantifies each layer’s sensitivity to different pruning granularities, and (2) an adaptive heterogeneous pruning method that uses this score select the appropriate pruning method for each layer. Experiments on LLaMA and other LLMs show that, at the 40% sparsity ratio, our method provides a controllable trade-off between performance and efficiency for pruned models.