Scholarlink

<p>The introduction of self-supervised pre-trained speech models (SPSMs) has accelerated research and practice within the speech community, enabling high-performance fine-tuning for downstream tasks like Automatic Speech Recognition (ASR). Prior studies have analyzed layer-level characteristics, but parameter-level insights remain limited, hindering model interpretability. This paper delves into the network parameter characteristics of SPSMs before and after fine-tuning for ASR. Using gradient parameter products as the metric, we identify a core set of parameters, termed "core regions," that significantly impact ASR performance, accounting for only approximately 1%-5% of the total model parameters. Experiments across seven languages from the Fleurs dataset, using the MMS-0.3B model, reveal that the core regions exhibit dimensional concentration and language invariance in distribution, as well as local fault tolerance and global decisiveness in performance. And, based on many characteristics of the core regions, an optimization method based on learnable gradient scaling factors was proposed. Our discovery of core regions offers valuable insight into the workings of SPSMs, and compared to traditional full parameter fine-tuning, the average WER of the proposed method is reduced by 3.35%, proving its effectiveness.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Investigation of core regions in self-supervised pre-trained speech models for automatic speech recognition

  • 期刊论文

  • 发表日期：

    2025年10月21日

查看全文
  • Yunpeng Liu,
  • Xukui Yang,
  • Dan Qu

摘要

The introduction of self-supervised pre-trained speech models (SPSMs) has accelerated research and practice within the speech community, enabling high-performance fine-tuning for downstream tasks like Automatic Speech Recognition (ASR). Prior studies have analyzed layer-level characteristics, but parameter-level insights remain limited, hindering model interpretability. This paper delves into the network parameter characteristics of SPSMs before and after fine-tuning for ASR. Using gradient parameter products as the metric, we identify a core set of parameters, termed "core regions," that significantly impact ASR performance, accounting for only approximately 1%-5% of the total model parameters. Experiments across seven languages from the Fleurs dataset, using the MMS-0.3B model, reveal that the core regions exhibit dimensional concentration and language invariance in distribution, as well as local fault tolerance and global decisiveness in performance. And, based on many characteristics of the core regions, an optimization method based on learnable gradient scaling factors was proposed. Our discovery of core regions offers valuable insight into the workings of SPSMs, and compared to traditional full parameter fine-tuning, the average WER of the proposed method is reduced by 3.35%, proving its effectiveness.