Scholarlink

<p>Large language models (LLMs) with multimodal capabilities may support automated assessment of cutaneous disease activity in dermatomyositis (DM). We evaluated the performance of Claude v3.5 Sonnet in scoring the Cutaneous Dermatomyositis Disease Area and Severity Index (CDASI) compared with expert rheumatologists. Thirty published DM cases with available clinical images were retrospectively analyzed. Two expert rheumatologists independently scored CDASI activity and damage domains. Claude v3.5 Sonnet assessed the same images using structured prompting based on CDASI definitions. Agreement was evaluated using intraclass correlation coefficients (ICCs) with a two-way random-effects model for absolute agreement. The LLM demonstrated better agreement for activity than for damage assessment. Global CDASI activity showed good agreement with expert raters (ICC 0.71, 95% CI 0.45–0.86), whereas damage reliability was lower (ICC 0.41, 95% CI 0.25–0.67). Agreement was moderate for activity-related domains including erythema (ICC 0.61), scaling (ICC 0.57), and erosions (ICC 0.57), while lower concordance was observed for chronic damage-related features such as poikiloderma (ICC 0.47, 95% CI 0.10–0.73). Hand assessments demonstrated the strongest performance, particularly for periungual changes (ICC 1.0, 95% CI 1.0–1.0) and global hand scores (ICC 0.95, 95% CI 0.91–0.97). The model also markedly reduced evaluation time compared with clinicians (42&#xa0;s vs. 8.4&#xa0;min per case), corresponding to an approximately 92% reduction in scoring time. Multimodal LLMs demonstrated promising agreement with expert raters for CDASI activity assessment and markedly improved efficiency, although lower reliability in damage domains reinforces the continued need for physician oversight.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Assessment of CDASI scoring by a multimodal large language model: a comparative study with expert assessors

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月8日

查看全文
  • Marco Fornaro,
  • Vincenzo Venerito,
  • Swapnasha Panigrahi,
  • Sara Sabbagh,
  • Florenzo Iannone,
  • Latika Gupta

摘要

Large language models (LLMs) with multimodal capabilities may support automated assessment of cutaneous disease activity in dermatomyositis (DM). We evaluated the performance of Claude v3.5 Sonnet in scoring the Cutaneous Dermatomyositis Disease Area and Severity Index (CDASI) compared with expert rheumatologists. Thirty published DM cases with available clinical images were retrospectively analyzed. Two expert rheumatologists independently scored CDASI activity and damage domains. Claude v3.5 Sonnet assessed the same images using structured prompting based on CDASI definitions. Agreement was evaluated using intraclass correlation coefficients (ICCs) with a two-way random-effects model for absolute agreement. The LLM demonstrated better agreement for activity than for damage assessment. Global CDASI activity showed good agreement with expert raters (ICC 0.71, 95% CI 0.45–0.86), whereas damage reliability was lower (ICC 0.41, 95% CI 0.25–0.67). Agreement was moderate for activity-related domains including erythema (ICC 0.61), scaling (ICC 0.57), and erosions (ICC 0.57), while lower concordance was observed for chronic damage-related features such as poikiloderma (ICC 0.47, 95% CI 0.10–0.73). Hand assessments demonstrated the strongest performance, particularly for periungual changes (ICC 1.0, 95% CI 1.0–1.0) and global hand scores (ICC 0.95, 95% CI 0.91–0.97). The model also markedly reduced evaluation time compared with clinicians (42 s vs. 8.4 min per case), corresponding to an approximately 92% reduction in scoring time. Multimodal LLMs demonstrated promising agreement with expert raters for CDASI activity assessment and markedly improved efficiency, although lower reliability in damage domains reinforces the continued need for physician oversight.