Scholarlink

Introduction <p>The aim of this study was to develop an interpretation pipeline for report generation for slit-lamp (SL) images, using vision-language models (VLMs).</p> Methods <p>An image-text alignment module (Bootstrapping Language-Image Pretraining [BLIP]) for report generation was developed using a dataset of SL images paired with medical reports from Zhejiang Provincial People’s Hospital (Zhaohui Hospital). These data were split into training and internal validation sets to finetune the BLIP frameworks (LLaVA and Qwen2.5-VL). A dataset from Bijie Hospital was used as an external validation dataset for the evaluation of the frameworks.</p> Results <p>The Zhaohui Hospital dataset included 1612 SL images and medical reports. The Bijie Hospital dataset included 100 SL images and medical reports. For the refined LLaVA framework, the BLEU scores (1–4) were 0.560–0.715, ROUGE-L score was 0.731, CIDEr score was 1.712, and SPICE score was 0.329. For the refined Qwen2.5-VL framework, the BLEU scores (1–4) were 0.536–0.696, ROUGE-L score was 0.696, CIDEr score was 1.729, and SPICE score was 0.208. Both models also showed good results in the classification of diseases. The overall accuracy was 0.87 in the refined LLaVA framework and 0.88 in the refined Qwen2.5-VL framework, with high accuracies (≥ 0.9) observed for eyelid disease, pterygium, glaucoma, corneal disease, and conjunctivitis. Interobserver agreement among ophthalmologists was substantial, with <i>κ</i> scores between 0.714 and 0.777. In the evaluation, the 100 reports generated by the LLaVA model demonstrated strong performance across all four metrics—correctness (2.72), completeness (2.79), harmlessness (2.88), and satisfaction (2.73)—with each scoring above 2.7. In comparison, the 100 reports produced by the Qwen model received slightly lower scores than LLaVA in correctness (2.63), completeness (2.70), and satisfaction (2.71).</p> Conclusions <p>This study introduced a framework for SL report generation, which enhanced ophthalmic image interpretation and highlighted the potential of VLMs to assist ophthalmologists and patients.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Report Generation System for Slit-Lamp Image Interpretation Using Vision-Language Models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月14日

查看全文
  • Xin Ye,
  • Yingjiao Shen,
  • Qian Chen,
  • Xiaoying He,
  • Xue Lu,
  • Andrzej Grzybowski,
  • Kai Jin,
  • Wenbin Xie

摘要

Introduction

The aim of this study was to develop an interpretation pipeline for report generation for slit-lamp (SL) images, using vision-language models (VLMs).

Methods

An image-text alignment module (Bootstrapping Language-Image Pretraining [BLIP]) for report generation was developed using a dataset of SL images paired with medical reports from Zhejiang Provincial People’s Hospital (Zhaohui Hospital). These data were split into training and internal validation sets to finetune the BLIP frameworks (LLaVA and Qwen2.5-VL). A dataset from Bijie Hospital was used as an external validation dataset for the evaluation of the frameworks.

Results

The Zhaohui Hospital dataset included 1612 SL images and medical reports. The Bijie Hospital dataset included 100 SL images and medical reports. For the refined LLaVA framework, the BLEU scores (1–4) were 0.560–0.715, ROUGE-L score was 0.731, CIDEr score was 1.712, and SPICE score was 0.329. For the refined Qwen2.5-VL framework, the BLEU scores (1–4) were 0.536–0.696, ROUGE-L score was 0.696, CIDEr score was 1.729, and SPICE score was 0.208. Both models also showed good results in the classification of diseases. The overall accuracy was 0.87 in the refined LLaVA framework and 0.88 in the refined Qwen2.5-VL framework, with high accuracies (≥ 0.9) observed for eyelid disease, pterygium, glaucoma, corneal disease, and conjunctivitis. Interobserver agreement among ophthalmologists was substantial, with κ scores between 0.714 and 0.777. In the evaluation, the 100 reports generated by the LLaVA model demonstrated strong performance across all four metrics—correctness (2.72), completeness (2.79), harmlessness (2.88), and satisfaction (2.73)—with each scoring above 2.7. In comparison, the 100 reports produced by the Qwen model received slightly lower scores than LLaVA in correctness (2.63), completeness (2.70), and satisfaction (2.71).

Conclusions

This study introduced a framework for SL report generation, which enhanced ophthalmic image interpretation and highlighted the potential of VLMs to assist ophthalmologists and patients.