Scholarlink

<p>Language is central to psychosis, making it a key focus for both clinical evaluation and research. Many automated linguistic metrics have been developed to analyze speech in psychosis, yet few studies have compared them systematically. In this study, we benchmarked the performance and robustness of 51 natural language processing (NLP) metrics using two datasets: one with German- and one with Russian-speaking participants. Metric performance was defined as the metrics’ capacity to reflect symptom severity and distinguish individuals with psychosis from healthy controls. The best-performing metrics tended to be robust across different clinical rating scales and speech elicitation tasks. Overall, verbosity metrics (e.g., word count) outperformed most other metrics across tasks and languages. Thus, verbosity-based metrics should be used as a baseline when evaluating cross-linguistic NLP metrics that are both clinically meaningful and useful for psychosis research.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cross-linguistic benchmarking of NLP metrics for psychosis research

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年8月3日

查看全文
  • D. Stein,
  • D. Bobrovskiy,
  • M. Stede,
  • C. Montag,
  • S. A. Just

摘要

Language is central to psychosis, making it a key focus for both clinical evaluation and research. Many automated linguistic metrics have been developed to analyze speech in psychosis, yet few studies have compared them systematically. In this study, we benchmarked the performance and robustness of 51 natural language processing (NLP) metrics using two datasets: one with German- and one with Russian-speaking participants. Metric performance was defined as the metrics’ capacity to reflect symptom severity and distinguish individuals with psychosis from healthy controls. The best-performing metrics tended to be robust across different clinical rating scales and speech elicitation tasks. Overall, verbosity metrics (e.g., word count) outperformed most other metrics across tasks and languages. Thus, verbosity-based metrics should be used as a baseline when evaluating cross-linguistic NLP metrics that are both clinically meaningful and useful for psychosis research.