Scholarlink

Background <p>Explainable AI (XAI) techniques like SHAP provide valuable insights into machine learning model predictions by quantifying feature contributions. However, interpreting these quantitative outputs remains unintuitive for many clinicians, hindering their practical adoption in clinical decision-making. This exploratory feasibility study aims to propose and evaluate a prompting framework designed to guide large language models (LLMs) in generating consistent, clinically relevant explanations from SHAP values.</p> Methods <p>We developed a structured zero-shot prompting framework incorporating variable definitions, safety principles, and a three-step reasoning process; (1) key risk factors, (2) prediction-outcome reconciliation, (3) clinical recommendations. This framework guided GPT-4 to explain SHAP-based predictions from an ICU extubation failure model (trained on MIMIC-III data). The framework’s performance was assessed using quantitative consistency metrics (entropy, accuracy) and a qualitative clinician survey (<i>n</i> = 7).</p> Results <p>The LLM achieved a fidelity of 0.783 and entropy of 0.226, indicating consistent structured reasoning. On a 5-point Likert scale, clinicians rated the LLM-generated explanations as helpful (mean: 3.94 ± 0.48) and safe (4.21 ± 0.68). However, critical care specialists assigned lower scores than non-critical care physicians (helpfulness: 3.48 vs. 4.29; safety: 3.45 vs. 4.79), suggesting domain-specific caution in perceived utility.</p> Conclusions <p>A structured prompting framework shows feasibility in leveraging LLMs to enhance the clinical interpretability of SHAP explanations. This methodological approach warrants further investigation and refinement for broader clinical application.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A prompt framework for enhancing LLM-based explainability of medical machine learning models: an intensive care unit application

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月26日

查看全文
  • Sujung Lee,
  • Won Ik Cho,
  • Youngrong Lee,
  • Duck Ju Kim,
  • Kyeng Hyun Nam,
  • Sangmin Lee,
  • Jungyo Suh,
  • Taehoon Ko

摘要

Background

Explainable AI (XAI) techniques like SHAP provide valuable insights into machine learning model predictions by quantifying feature contributions. However, interpreting these quantitative outputs remains unintuitive for many clinicians, hindering their practical adoption in clinical decision-making. This exploratory feasibility study aims to propose and evaluate a prompting framework designed to guide large language models (LLMs) in generating consistent, clinically relevant explanations from SHAP values.

Methods

We developed a structured zero-shot prompting framework incorporating variable definitions, safety principles, and a three-step reasoning process; (1) key risk factors, (2) prediction-outcome reconciliation, (3) clinical recommendations. This framework guided GPT-4 to explain SHAP-based predictions from an ICU extubation failure model (trained on MIMIC-III data). The framework’s performance was assessed using quantitative consistency metrics (entropy, accuracy) and a qualitative clinician survey (n = 7).

Results

The LLM achieved a fidelity of 0.783 and entropy of 0.226, indicating consistent structured reasoning. On a 5-point Likert scale, clinicians rated the LLM-generated explanations as helpful (mean: 3.94 ± 0.48) and safe (4.21 ± 0.68). However, critical care specialists assigned lower scores than non-critical care physicians (helpfulness: 3.48 vs. 4.29; safety: 3.45 vs. 4.79), suggesting domain-specific caution in perceived utility.

Conclusions

A structured prompting framework shows feasibility in leveraging LLMs to enhance the clinical interpretability of SHAP explanations. This methodological approach warrants further investigation and refinement for broader clinical application.