Scholarlink

Objective <p>To systematically evaluate the diagnostic, classification, and treatment decision-making performance of large language models (LLMs) in dry eye disease (DED) and compare their performance with that of junior ophthalmologists to assess their feasibility as clinical support tools.</p> Methods <p>One hundred standardized DED cases from Fuzhou University Affiliated Provincial Hospital (Aug 2023–Aug 2025) were analyzed. Four LLMs (ChatGPT-4o, DeepSeek-V3, Gemini-2.5-Pro, and ERNIE Bot-4.5-turbo) underwent an initial 20-item test with a 95% accuracy threshold. Only models meeting this criterion advanced to case-based evaluation. Diagnostic and therapeutic outputs were rated by senior ophthalmologists using the Global Quality Score (GQS) and a clinical safety score. Meanwhile, the response times of both the LLMs and the junior ophthalmologists were recorded to evaluate their respective efficiency.</p> Results <p>ChatGPT-4o, DeepSeek-V3, and Gemini-2.5-Pro met the screening threshold. In determining treatment necessity, the accuracies of the LLMs (96–99%) were comparable to those of junior ophthalmologists (97%). For DED classification, DeepSeek-V3 achieved the highest accuracy (92%) and agreement with experts (kappa = 0.80), outperforming ChatGPT-4o and junior ophthalmologists (71%, kappa ≈ 0.53; <i>p</i> &lt; 0.01). Gemini-2.5-Pro showed strong performance (accuracy 89%), the highest GQS (4.87 ± 0.37), and the best safety rating, with 89% of its outputs rated as “good”. The decision efficiency of the LLMs was significantly higher (<i>p</i> ≤ 0.001), with response times (16–36&#xa0;s) much shorter than those of junior ophthalmologists (315 ± 57&#xa0;s).</p> Conclusions <p>Gemini-2.5-Pro and DeepSeek-V3 demonstrated high accuracy, safety, and efficiency in case-based DED management, showing strong potential as auxiliary tools to enhance clinical decision-making and support less experienced clinicians.</p> Trial registration <p>Not applicable. This study is a retrospective observational study and does not involve any intervention requiring trial registration.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Application of large language models in clinical decision-making for dry eye disease

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月26日

查看全文
  • Haoqiang Cui,
  • Yuyang Yang,
  • Taichen Lai,
  • Xuemei Zhang,
  • Kunhong Xiao,
  • Li Liu,
  • Ziyi Qi,
  • Catherine Jan,
  • Yuqing Wang,
  • Beiou Zhang,
  • Jiahao Liu,
  • Zhuoting Zhu,
  • Yan Huang,
  • Li Li

摘要

Objective

To systematically evaluate the diagnostic, classification, and treatment decision-making performance of large language models (LLMs) in dry eye disease (DED) and compare their performance with that of junior ophthalmologists to assess their feasibility as clinical support tools.

Methods

One hundred standardized DED cases from Fuzhou University Affiliated Provincial Hospital (Aug 2023–Aug 2025) were analyzed. Four LLMs (ChatGPT-4o, DeepSeek-V3, Gemini-2.5-Pro, and ERNIE Bot-4.5-turbo) underwent an initial 20-item test with a 95% accuracy threshold. Only models meeting this criterion advanced to case-based evaluation. Diagnostic and therapeutic outputs were rated by senior ophthalmologists using the Global Quality Score (GQS) and a clinical safety score. Meanwhile, the response times of both the LLMs and the junior ophthalmologists were recorded to evaluate their respective efficiency.

Results

ChatGPT-4o, DeepSeek-V3, and Gemini-2.5-Pro met the screening threshold. In determining treatment necessity, the accuracies of the LLMs (96–99%) were comparable to those of junior ophthalmologists (97%). For DED classification, DeepSeek-V3 achieved the highest accuracy (92%) and agreement with experts (kappa = 0.80), outperforming ChatGPT-4o and junior ophthalmologists (71%, kappa ≈ 0.53; p < 0.01). Gemini-2.5-Pro showed strong performance (accuracy 89%), the highest GQS (4.87 ± 0.37), and the best safety rating, with 89% of its outputs rated as “good”. The decision efficiency of the LLMs was significantly higher (p ≤ 0.001), with response times (16–36 s) much shorter than those of junior ophthalmologists (315 ± 57 s).

Conclusions

Gemini-2.5-Pro and DeepSeek-V3 demonstrated high accuracy, safety, and efficiency in case-based DED management, showing strong potential as auxiliary tools to enhance clinical decision-making and support less experienced clinicians.

Trial registration

Not applicable. This study is a retrospective observational study and does not involve any intervention requiring trial registration.