Scholarlink

<p>Evaluating large language models (LLMs) in the mental health domain presents distinct challenges due to the subtle, context-dependent, and subjective nature of psychological symptoms. We introduce <i>PsyEval</i>, a benchmark specifically designed to evaluate LLMs in mental health-related tasks across three core dimensions: knowledge, diagnosis, and emotional support. PsyEval is constructed to reflect the complexity of mental health scenarios and provides a structured framework for assessing model performance within this sensitive domain. Using PsyEval, we evaluate eleven advanced LLMs with different prompting strategies to investigate how prompting affects their responses. The results reveal considerable gaps in LLMs’ current ability to reason accurately and respond appropriately in mental health contexts, while also indicating promising directions for future model enhancement.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

PsyEval: a comprehensive large language model evaluation benchmark for mental health

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月10日

查看全文
  • Haoan Jin,
  • Chen Siyuan,
  • Dilawaier Dilixiati,
  • Yewei Jiang,
  • Kenny Q. Zhu,
  • Mengyue Wu

摘要

Evaluating large language models (LLMs) in the mental health domain presents distinct challenges due to the subtle, context-dependent, and subjective nature of psychological symptoms. We introduce PsyEval, a benchmark specifically designed to evaluate LLMs in mental health-related tasks across three core dimensions: knowledge, diagnosis, and emotional support. PsyEval is constructed to reflect the complexity of mental health scenarios and provides a structured framework for assessing model performance within this sensitive domain. Using PsyEval, we evaluate eleven advanced LLMs with different prompting strategies to investigate how prompting affects their responses. The results reveal considerable gaps in LLMs’ current ability to reason accurately and respond appropriately in mental health contexts, while also indicating promising directions for future model enhancement.