Scholarlink

Purpose <p>This study evaluates the performance of four large language models (LLMs) in classifying malignant lymphoma stages using the Lugano classification from free-text FDG-PET reports in Japanese Specifically, we assess GPT-4o, Claude 3.5 Sonnet, Llama 3 70B, and Gemma 2 27B in their ability interpret unstructured radiology texts.</p> Materials and methods <p>In a retrospective single-center study, 80 patients who underwent staging FDG-PET/CT for malignant lymphoma were included. The “Findings” sections of their reports were analyzed without pre-processing. Each LLM assigned Lugano stages based on these reports. Performance was compared to reference standard stages determined by expert radiologists. Statistical analyses involved overall accuracy, weighted kappa for agreement.</p> Results <p>GPT-4o achieved the highest accuracy at 75% (60/80 cases) with substantial agreement (weighted kappa κ = 0.801). Claude 3.5 Sonnet had 61.3% accuracy (49/80, κ = 0.763). Gemma 2 27B and Llama 3 70B showed accuracies of 58.8% and 57.5%, respectively, all indicating substantial agreement.</p> Conclusion <p>GPT-4o outperformed other LLMs in assigning Lugano classification from Japanese FDG-PET free-text reports. This demonstrated the potential of advanced LLMs to interpret clinical texts. While the immediate clinical utility of automatically predicting a Lugano stage from an existing report may be limited, these results highlight the value of LLMs for understanding and standardizing free-text radiology data.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Assessing large language models for Lugano classification of malignant lymphoma in Japanese FDG-PET reports

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年3月10日

查看全文
  • Rintaro Ito,
  • Keita Kato,
  • Kosuke Nanataki,
  • Yumi Abe,
  • Hiroshi Ogawa,
  • Ryogo Minamimoto,
  • Katsuhiko Kato,
  • Toshiaki Taoka,
  • Shinji Naganawa

摘要

Purpose

This study evaluates the performance of four large language models (LLMs) in classifying malignant lymphoma stages using the Lugano classification from free-text FDG-PET reports in Japanese Specifically, we assess GPT-4o, Claude 3.5 Sonnet, Llama 3 70B, and Gemma 2 27B in their ability interpret unstructured radiology texts.

Materials and methods

In a retrospective single-center study, 80 patients who underwent staging FDG-PET/CT for malignant lymphoma were included. The “Findings” sections of their reports were analyzed without pre-processing. Each LLM assigned Lugano stages based on these reports. Performance was compared to reference standard stages determined by expert radiologists. Statistical analyses involved overall accuracy, weighted kappa for agreement.

Results

GPT-4o achieved the highest accuracy at 75% (60/80 cases) with substantial agreement (weighted kappa κ = 0.801). Claude 3.5 Sonnet had 61.3% accuracy (49/80, κ = 0.763). Gemma 2 27B and Llama 3 70B showed accuracies of 58.8% and 57.5%, respectively, all indicating substantial agreement.

Conclusion

GPT-4o outperformed other LLMs in assigning Lugano classification from Japanese FDG-PET free-text reports. This demonstrated the potential of advanced LLMs to interpret clinical texts. While the immediate clinical utility of automatically predicting a Lugano stage from an existing report may be limited, these results highlight the value of LLMs for understanding and standardizing free-text radiology data.