Scholarlink

Automatic Speech Recognition (ASR) holds significant potential for reducing the workload of medical staff, primarily by automating documentation tasks. While numerous benchmarks exist for the English language, specific evaluations for the German-speaking medical context – particularly those considering dialects and accents – are still lacking. In this paper, we present a dataset of simulated doctor-patient conversations and evaluate a total of 29 different ASR models. Our test set encompasses open-weight models from the Whisper, Voxtral, and Wav2Vec2 families, as well as commercial state-of-the-art APIs (AssemblyAI, Deepgram). For evaluation, we employ three distinct metrics (WER, CER, BLEU) and provide an outlook on qualitative semantic analysis. The results reveal significant performance discrepancies among the models: while top-tier systems achieve very low Word Error Rates (WER), other models exhibit considerably higher error rates, especially when processing medical terminology or dialect-heavy speech.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Benchmarking ASR Models in German Medical Contexts: A Performance Analysis Using Anamnesis Conversations

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Thomas Schuster,
  • Julius Trögele,
  • Nico Döring,
  • Robin Krüger,
  • Mathieu Hoffmann,
  • Holger Friedrich

摘要

Automatic Speech Recognition (ASR) holds significant potential for reducing the workload of medical staff, primarily by automating documentation tasks. While numerous benchmarks exist for the English language, specific evaluations for the German-speaking medical context – particularly those considering dialects and accents – are still lacking. In this paper, we present a dataset of simulated doctor-patient conversations and evaluate a total of 29 different ASR models. Our test set encompasses open-weight models from the Whisper, Voxtral, and Wav2Vec2 families, as well as commercial state-of-the-art APIs (AssemblyAI, Deepgram). For evaluation, we employ three distinct metrics (WER, CER, BLEU) and provide an outlook on qualitative semantic analysis. The results reveal significant performance discrepancies among the models: while top-tier systems achieve very low Word Error Rates (WER), other models exhibit considerably higher error rates, especially when processing medical terminology or dialect-heavy speech.