Objectives To compare understandability, readability, and communication characteristics of radiology report explanations generated by three freely accessible large language models—ChatGPT, Gemini, and Copilot—based on a standardized prompt, as assessed by expert reviewers. Materials and methods In this retrospective single-center study, 100 anonymized radiology reports were randomly selected from five subspecialties. Each report was submitted to ChatGPT (GPT-3.5), Gemini, and Copilot between May 23 and May 30, 2025, using the prompt, “Can you explain my radiology report?”. Responses were evaluated for medical correctness on a 3-point scale (0–2), understandability using the patient education materials assessment tool for understandability (PEMAT-U), and readability using Flesch Reading Ease (FRE), Automated Readability Index (ARI), and Gunning Fog Index (GFI). Communicative features—including uncertainty language, patient guidance, and clinical suggestions—were also assessed. Anxiety-inducing potential was rated on a 3-point Likert scale. Results All models demonstrated high medical correctness (mean: 1.97 ± 0.17/2). ChatGPT produced the most readable (FRE: 60.33 ± 3.65; ARI: 9.66 ± 1.01; GFI: 9.1 ± 1.04) and understandable (PEMAT-U: 89.58 ± 3.90%) responses (p < 0.01). Copilot included the most uncertainty language (1.62 ± 0.62) and clinical suggestions (1.69 ± 0.60), while Gemini provided the strongest patient guidance (1.62 ± 0.58) (all p < 0.01). Only Copilot showed subspecialty-related variation in readability (GFI; p = 0.048). Anxiety potential was low across all models (mean: 0.07 ± 0.33). Conclusion ChatGPT offered superior readability and understandability. Copilot delivered more clinical detail with cautious language, while Gemini emphasized patient-centered guidance. These differences support context-specific use of language models in radiology communication. Critical relevance statement This study shows that freely accessible large language models produce radiology report explanations with varying levels of readability, understandability, and communication quality. Expert-based findings may help inform future strategies to optimize patient-facing applications of AI in radiological communication. Key Points <UnorderedList Mark="Bullet"> <ItemContent> This study compared how freely available AI chatbots respond to patient queries about radiology reports. </ItemContent> <ItemContent> Significant differences were found in understandability, readability, patient guidance, and use of uncertainty or clinical suggestions. </ItemContent> <ItemContent> Findings support context-specific use of AI tools to improve radiology communication and patient understanding. </ItemContent> </UnorderedList> Graphical Abstract

Comparative Evaluation of Large Language Models in Explaining Radiology Reports: Expert Assessment of Readability, Understandability, and Communication Features

期刊论文
出版模式：

开放获取
发表日期：

2025年10月29日

查看全文

Ahmet Bozer,
Yeliz Pekçevik

摘要

Objectives

To compare understandability, readability, and communication characteristics of radiology report explanations generated by three freely accessible large language models—ChatGPT, Gemini, and Copilot—based on a standardized prompt, as assessed by expert reviewers.

Materials and methods

In this retrospective single-center study, 100 anonymized radiology reports were randomly selected from five subspecialties. Each report was submitted to ChatGPT (GPT-3.5), Gemini, and Copilot between May 23 and May 30, 2025, using the prompt, “Can you explain my radiology report?”. Responses were evaluated for medical correctness on a 3-point scale (0–2), understandability using the patient education materials assessment tool for understandability (PEMAT-U), and readability using Flesch Reading Ease (FRE), Automated Readability Index (ARI), and Gunning Fog Index (GFI). Communicative features—including uncertainty language, patient guidance, and clinical suggestions—were also assessed. Anxiety-inducing potential was rated on a 3-point Likert scale.

Results

All models demonstrated high medical correctness (mean: 1.97 ± 0.17/2). ChatGPT produced the most readable (FRE: 60.33 ± 3.65; ARI: 9.66 ± 1.01; GFI: 9.1 ± 1.04) and understandable (PEMAT-U: 89.58 ± 3.90%) responses (p < 0.01). Copilot included the most uncertainty language (1.62 ± 0.62) and clinical suggestions (1.69 ± 0.60), while Gemini provided the strongest patient guidance (1.62 ± 0.58) (all p < 0.01). Only Copilot showed subspecialty-related variation in readability (GFI; p = 0.048). Anxiety potential was low across all models (mean: 0.07 ± 0.33).

Conclusion

ChatGPT offered superior readability and understandability. Copilot delivered more clinical detail with cautious language, while Gemini emphasized patient-centered guidance. These differences support context-specific use of language models in radiology communication.

Critical relevance statement

This study shows that freely accessible large language models produce radiology report explanations with varying levels of readability, understandability, and communication quality. Expert-based findings may help inform future strategies to optimize patient-facing applications of AI in radiological communication.

Key Points

This study compared how freely available AI chatbots respond to patient queries about radiology reports.

Significant differences were found in understandability, readability, patient guidance, and use of uncertainty or clinical suggestions.

Findings support context-specific use of AI tools to improve radiology communication and patient understanding.

Graphical Abstract