Scholarlink

<p>Building a robust ASR system for dysarthric speech is challenging due to its irregular articulation, phonation, and prosody, which differ markedly from normal speech. The advent of transformer-based end-to-end systems has notably improved ASR performance for normal speech recognition tasks. In particular, adapting pre-trained transformer-based ASR models trained on large amounts of similar data exhibits superior performance compared to models trained from scratch. However, the efficacy of such pre-trained models is limited to normal speech, whereas their performance in dysarthric speech recognition remains uncertain. This paper explores the extension of these advances to dysarthric speech recognition. We investigate fine-tuning the Wav2vec2 pretrained model on dysarthric speech data, with a focus on two factors: severity level (Low, Medium, High, or Very High) and speaker independence. The initial fine-tuning experiments were severity-specific, conducted with a focus on dysarthria severity levels. From these experiments, it was observed that incorporating data from higher severity cases into the training process significantly improves the model’s ability to recognize lower severity instances. The severity-specific fine-tuning showed an overall relative improvement of 32.93% and 31.42% in TORGO and UASpeech speech databases, respectively, with significance tests yielding <i>p</i>-values of 0.016 and 0.004 compared to the baseline inference result. On average, the severity-specific fine-tuned models achieved WERs of 40.48% and 51.79% and CERs of 25.99% and 41.18% on the TORGO and UASpeech databases, respectively, and this improvement persisted under four real-world noise conditions (Birds, Nature, Thunder, Babble) at SNRs of 15, 10, and 5 dB, demonstrating robustness compared to the baseline. Furthermore, the study of mix-severity cases showed highly effective results on both TORGO and UASpeech dysarthric speech datasets, with average relative improvements of 55.99% and 61.03%, respectively in comparison to the baseline. Examining the mix-severity setting in the speaker-independent context resulted in mean Word Error Rates (WERs) of 26.56% and 29.64% for the TORGO and UASpeech datasets, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Addressing Dysarthric Speech Variability with Severity-Aware Fine-Tuning of Transformer-Based Wav2vec2 ASR Models

  • 期刊论文

  • 发表日期：

    2026年2月9日

查看全文
  • Paban Sapkota,
  • Hemant Kumar Kathania,
  • Shrikanth Narayanan,
  • Sudarsana Reddy Kadiri

摘要

Building a robust ASR system for dysarthric speech is challenging due to its irregular articulation, phonation, and prosody, which differ markedly from normal speech. The advent of transformer-based end-to-end systems has notably improved ASR performance for normal speech recognition tasks. In particular, adapting pre-trained transformer-based ASR models trained on large amounts of similar data exhibits superior performance compared to models trained from scratch. However, the efficacy of such pre-trained models is limited to normal speech, whereas their performance in dysarthric speech recognition remains uncertain. This paper explores the extension of these advances to dysarthric speech recognition. We investigate fine-tuning the Wav2vec2 pretrained model on dysarthric speech data, with a focus on two factors: severity level (Low, Medium, High, or Very High) and speaker independence. The initial fine-tuning experiments were severity-specific, conducted with a focus on dysarthria severity levels. From these experiments, it was observed that incorporating data from higher severity cases into the training process significantly improves the model’s ability to recognize lower severity instances. The severity-specific fine-tuning showed an overall relative improvement of 32.93% and 31.42% in TORGO and UASpeech speech databases, respectively, with significance tests yielding p-values of 0.016 and 0.004 compared to the baseline inference result. On average, the severity-specific fine-tuned models achieved WERs of 40.48% and 51.79% and CERs of 25.99% and 41.18% on the TORGO and UASpeech databases, respectively, and this improvement persisted under four real-world noise conditions (Birds, Nature, Thunder, Babble) at SNRs of 15, 10, and 5 dB, demonstrating robustness compared to the baseline. Furthermore, the study of mix-severity cases showed highly effective results on both TORGO and UASpeech dysarthric speech datasets, with average relative improvements of 55.99% and 61.03%, respectively in comparison to the baseline. Examining the mix-severity setting in the speaker-independent context resulted in mean Word Error Rates (WERs) of 26.56% and 29.64% for the TORGO and UASpeech datasets, respectively.