Scholarlink

Objectives <p>To perform an in-depth evaluation of the diagnostic test accuracy of a commercially available AI tool for assistance in fracture detection on radiographs.</p> Materials and methods <p>This retrospective study included consecutive patients with trauma radiographs at seven Danish hospitals. The AI output was evaluated using the clinical radiologic report as a reference standard for a binary fracture outcome. The report is based on assessments by an emergency physician, a senior orthopedic surgeon, and a radiology expert. Sensitivity, specificity, positive- and negative predictive values were calculated. Sensitivity and specificity were additionally stratified for children, degenerative disease, metal, old fractures, casting, obvious fractures, and inter-hospital differences. Bone-wise sensitivity and specificity were assessed for multiple fracture cases and individual bones.</p> Results <p>The study sample consisted of 2783 patients (median age 38 years, IQR, 21, 64, 1443 female), and 948 (34%) had the target finding. The AI tool demonstrated an overall sensitivity of 89% (95% CI: 87%–91%) and specificity of 88% (95% CI: 86%–89%). The specificity was 57% (95% CI: 49%–65%) in examinations with old fractures. Bone-wise sensitivity for carpal fractures ranged from other carpals 25% (95% CI: 1%–81%] to triquetrum 75% (95% CI: 43%–95%). Tarsal fractures ranged from medial cuneiform 0% (95% CI: 0%–60%) to talus 53% (95% CI: 27%–79%).</p> Conclusion <p>The AI tool demonstrated high overall diagnostic accuracy and performed robustly across most specific situations. However, specificity was substantially reduced in the presence of old fractures. The bone-wise analysis showed great variability, with a pattern of poor accuracy for short, irregular bones.</p> Key Points <p><Emphasis Type="BoldItalic">Question</Emphasis> <i>Can a commercially available AI tool reliably detect fractures across anatomical regions, confounding factors, and individual bones -and are there patterns in diagnostic limitations</i>?</p> <p><Emphasis Type="BoldItalic">Findings</Emphasis> <i>The AI tool achieved 89% sensitivity and 88% specificity with consistent accuracy across subgroups. However, accuracy dropped for old fractures and irregular short bones</i>.</p> <p><Emphasis Type="BoldItalic">Clinical relevance</Emphasis> <i>Despite broad regulatory approval, AI fracture tools may overlook clinically relevant weaknesses. Our in-depth evaluation highlights limitations, guiding responsible clinical use and future research to support safe AI implementation in radiology and informed medicolegal regulation</i>.</p> Graphical Abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Independent bone-level diagnostic accuracy study of an AI tool for detecting appendicular skeletal fractures on radiographs

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月17日

查看全文
  • Frederik Jager Bruun,
  • Felix C. Müller,
  • Janus Uhd Nybing,
  • Philip Hansen,
  • Kasper Kjærulf Gosvig,
  • Mikael Ploug Boesen,
  • Mathias Willadsen Brejnebøl

摘要

Objectives

To perform an in-depth evaluation of the diagnostic test accuracy of a commercially available AI tool for assistance in fracture detection on radiographs.

Materials and methods

This retrospective study included consecutive patients with trauma radiographs at seven Danish hospitals. The AI output was evaluated using the clinical radiologic report as a reference standard for a binary fracture outcome. The report is based on assessments by an emergency physician, a senior orthopedic surgeon, and a radiology expert. Sensitivity, specificity, positive- and negative predictive values were calculated. Sensitivity and specificity were additionally stratified for children, degenerative disease, metal, old fractures, casting, obvious fractures, and inter-hospital differences. Bone-wise sensitivity and specificity were assessed for multiple fracture cases and individual bones.

Results

The study sample consisted of 2783 patients (median age 38 years, IQR, 21, 64, 1443 female), and 948 (34%) had the target finding. The AI tool demonstrated an overall sensitivity of 89% (95% CI: 87%–91%) and specificity of 88% (95% CI: 86%–89%). The specificity was 57% (95% CI: 49%–65%) in examinations with old fractures. Bone-wise sensitivity for carpal fractures ranged from other carpals 25% (95% CI: 1%–81%] to triquetrum 75% (95% CI: 43%–95%). Tarsal fractures ranged from medial cuneiform 0% (95% CI: 0%–60%) to talus 53% (95% CI: 27%–79%).

Conclusion

The AI tool demonstrated high overall diagnostic accuracy and performed robustly across most specific situations. However, specificity was substantially reduced in the presence of old fractures. The bone-wise analysis showed great variability, with a pattern of poor accuracy for short, irregular bones.

Key Points

Question Can a commercially available AI tool reliably detect fractures across anatomical regions, confounding factors, and individual bones -and are there patterns in diagnostic limitations?

Findings The AI tool achieved 89% sensitivity and 88% specificity with consistent accuracy across subgroups. However, accuracy dropped for old fractures and irregular short bones.

Clinical relevance Despite broad regulatory approval, AI fracture tools may overlook clinically relevant weaknesses. Our in-depth evaluation highlights limitations, guiding responsible clinical use and future research to support safe AI implementation in radiology and informed medicolegal regulation.

Graphical Abstract