Scholarlink

<p>Mental health disorders pose a growing public health concern in the Arab world, highlighting the need for accessible diagnostic and intervention tools. Large language models (LLMs) offer a promising approach, but their application in Arabic contexts faces challenges, including limited labeled datasets and linguistic complexity. This study comprehensively evaluates eight LLMs (both general multilingual and bilingual models) using 26 evaluation files derived from cross-translating 13 mental health datasets: six native-Arabic social-media sets and seven native-English sets, covering binary and multiclass classification, multiple-choice and severity tasks, with performance assessed by balanced accuracy and scale-normalized mean absolute error. We investigate the impact of prompt design, language configuration (native Arabic vs. translated English and vice versa), and few-shot prompting on diagnostic performance. Prompt engineering significantly influences LLM scores, mainly through improved instruction following; our structured prompt outperforms a less structured variant on multiclass tasks by an average of 14.5% balanced accuracy. Language effects are modest overall, but model selection is crucial: Phi-3.5 MoE excels in balanced accuracy, particularly on binary classification, while Mistral NeMo shows the lowest mean absolute error for severity prediction. Few-shot prompting consistently improves results, with substantial gains for GPT-4o Mini on multiclass classification (1.58<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\times \)</EquationSource> <EquationSource Format="MATHML"><math> <mo>×</mo> </math></EquationSource> </InlineEquation> average accuracy boost). These findings underscore the importance of prompt optimization, multilingual analysis, and few-shot learning, and we provide a ranking of models along with a dataset-difficulty analysis to guide culturally sensitive LLM-based mental health tools for Arabic-speaking populations.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Benchmarking Large Language Models for Arabic Mental Health Screening

  • 期刊论文

  • 发表日期：

    2026年2月21日

查看全文
  • Noureldin Zahran,
  • Aya E. Fouda,
  • Radwa J. Hanafy,
  • Mohammed E. Fouda

摘要

Mental health disorders pose a growing public health concern in the Arab world, highlighting the need for accessible diagnostic and intervention tools. Large language models (LLMs) offer a promising approach, but their application in Arabic contexts faces challenges, including limited labeled datasets and linguistic complexity. This study comprehensively evaluates eight LLMs (both general multilingual and bilingual models) using 26 evaluation files derived from cross-translating 13 mental health datasets: six native-Arabic social-media sets and seven native-English sets, covering binary and multiclass classification, multiple-choice and severity tasks, with performance assessed by balanced accuracy and scale-normalized mean absolute error. We investigate the impact of prompt design, language configuration (native Arabic vs. translated English and vice versa), and few-shot prompting on diagnostic performance. Prompt engineering significantly influences LLM scores, mainly through improved instruction following; our structured prompt outperforms a less structured variant on multiclass tasks by an average of 14.5% balanced accuracy. Language effects are modest overall, but model selection is crucial: Phi-3.5 MoE excels in balanced accuracy, particularly on binary classification, while Mistral NeMo shows the lowest mean absolute error for severity prediction. Few-shot prompting consistently improves results, with substantial gains for GPT-4o Mini on multiclass classification (1.58 \(\times \) × average accuracy boost). These findings underscore the importance of prompt optimization, multilingual analysis, and few-shot learning, and we provide a ranking of models along with a dataset-difficulty analysis to guide culturally sensitive LLM-based mental health tools for Arabic-speaking populations.