Scholarlink

Introduction <p>Pneumothorax is a potentially life-threatening clinical condition requiring rapid diagnosis and intervention. It may present with symptoms such as dyspnoea, cough, chest pain, tachycardia, and tachypnoea. Heavy workload and technical limitations of chest radiographs may necessitate support for clinicians during the evaluation process. Recently, large language models (LLMs) have increased interest in using their ability to process both text and medical images as supportive tools in diagnostic processes. This study analyzed the performance of two widely used multimodal LLMs in diagnosing pneumothorax in a real clinical setting.</p> Materials and methods <p>Clinical information and anonymized chest X-rays were presented to both models using an eight-question evaluation file. The responses obtained were compared with the consensus assessment of two thoracic surgeons, considered the gold standard.</p> Results <p>Pneumothorax was detected in 30 of the total 240 patients (12.5%). ChatGPT demonstrated 100% sensitivity, 91.4% specificity, 62.5% positive predictive value (PPV), and 100% negative predictive value (NPV) in detecting pneumothorax. Gemini had a sensitivity of 70.0%, specificity of 92.9%, PPV of 58.3%, and NPV of 95.6%. The performance of both models in determining the side of pneumothorax was markedly low: ChatGPT had a sensitivity of 14.3%, and Gemini had a sensitivity of 10.0%. When it involved determining the type of pneumothorax, ChatGPT demonstrated a sensitivity of 36.7% (95% CI: 21.9–54.5), while Gemini showed a sensitivity of 60.0% (95% CI: 42.3–75.4). Both models exhibited high specificity (&gt; 91%). When assessing the need for tube thoracostomy, ChatGPT showed 57.7% sensitivity and 94.4% specificity, whereas Gemini showed 46.2% sensitivity and 95.8% specificity. NPV was high and PPV was low in both models.</p> Conclusion <p>Multimodal LLMs demonstrated high sensitivity in ruling out pneumothorax; however, limitations in agreement, localisation, and management decisions restrict their role to cautious, supportive use under specialist supervision.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluation of multimodal large language models for pneumothorax assessment in real-world clinical scenarios

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月5日

查看全文
  • Kadir Baturhan Ciflik,
  • Busra Ozdemir Ciflik

摘要

Introduction

Pneumothorax is a potentially life-threatening clinical condition requiring rapid diagnosis and intervention. It may present with symptoms such as dyspnoea, cough, chest pain, tachycardia, and tachypnoea. Heavy workload and technical limitations of chest radiographs may necessitate support for clinicians during the evaluation process. Recently, large language models (LLMs) have increased interest in using their ability to process both text and medical images as supportive tools in diagnostic processes. This study analyzed the performance of two widely used multimodal LLMs in diagnosing pneumothorax in a real clinical setting.

Materials and methods

Clinical information and anonymized chest X-rays were presented to both models using an eight-question evaluation file. The responses obtained were compared with the consensus assessment of two thoracic surgeons, considered the gold standard.

Results

Pneumothorax was detected in 30 of the total 240 patients (12.5%). ChatGPT demonstrated 100% sensitivity, 91.4% specificity, 62.5% positive predictive value (PPV), and 100% negative predictive value (NPV) in detecting pneumothorax. Gemini had a sensitivity of 70.0%, specificity of 92.9%, PPV of 58.3%, and NPV of 95.6%. The performance of both models in determining the side of pneumothorax was markedly low: ChatGPT had a sensitivity of 14.3%, and Gemini had a sensitivity of 10.0%. When it involved determining the type of pneumothorax, ChatGPT demonstrated a sensitivity of 36.7% (95% CI: 21.9–54.5), while Gemini showed a sensitivity of 60.0% (95% CI: 42.3–75.4). Both models exhibited high specificity (> 91%). When assessing the need for tube thoracostomy, ChatGPT showed 57.7% sensitivity and 94.4% specificity, whereas Gemini showed 46.2% sensitivity and 95.8% specificity. NPV was high and PPV was low in both models.

Conclusion

Multimodal LLMs demonstrated high sensitivity in ruling out pneumothorax; however, limitations in agreement, localisation, and management decisions restrict their role to cautious, supportive use under specialist supervision.