Scholarlink

Effective healthcare communication in linguistically diverse and resource-constrained environments like India, home to 22 official languages, remains a significant challenge, often affecting the quality and equity of care delivery. To address this, we introduce IndiHealthBench, a multilingual benchmark focused on medical translation, comprising 11,000 parallel sentence pairs spanning 13 \(^\textrm{th}\) Indian languages. Designed to evaluate the capabilities of both general-purpose and medically fine-tuned Large Language Models (LLMs), the benchmark facilitates bi-directional translation tasks between English and Indian languages, with a particular focus on clinical terminology and semantic fidelity. Through comprehensive metric-based evaluations and detailed qualitative analyses, we uncover key linguistic and translation challenges encountered by LLMs in clinical settings. Our findings reveal that medical-domain LLMs consistently outperform general models, particularly in morphologically complex and low-resource languages. This work offers critical insights into building robust, inclusive, and trustworthy multilingual healthcare systems for underserved populations.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

IndiHealthBench: Evaluating LLMs for Clinical Translation Across Indian Linguistic Diversity

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Pooja Singh,
  • Jay Saraf,
  • Aashrith Sharma,
  • Siddhant Ujjain,
  • Paridhi Jain,
  • Sandeep Kumar

摘要

Effective healthcare communication in linguistically diverse and resource-constrained environments like India, home to 22 official languages, remains a significant challenge, often affecting the quality and equity of care delivery. To address this, we introduce IndiHealthBench, a multilingual benchmark focused on medical translation, comprising 11,000 parallel sentence pairs spanning 13 \(^\textrm{th}\) Indian languages. Designed to evaluate the capabilities of both general-purpose and medically fine-tuned Large Language Models (LLMs), the benchmark facilitates bi-directional translation tasks between English and Indian languages, with a particular focus on clinical terminology and semantic fidelity. Through comprehensive metric-based evaluations and detailed qualitative analyses, we uncover key linguistic and translation challenges encountered by LLMs in clinical settings. Our findings reveal that medical-domain LLMs consistently outperform general models, particularly in morphologically complex and low-resource languages. This work offers critical insights into building robust, inclusive, and trustworthy multilingual healthcare systems for underserved populations.