Scholarlink

Background <p>We developed a multi-modal Transformer framework integrating knee radiographs with clinical covariates to enable automated, objective, and generalizable ordinal Kellgren-Lawrence (KL) grading.</p> Methods <p>A total of 2,703 anteroposterior knee radiographs were retrospectively collected from three independent medical centers (January 2018 - December 2024). Data from two centers (<i>n</i> = 1,953) were used for model development and internal five-fold stratified cross-validation, while the third center (<i>n</i> = 750) served as an independent external test set. The proposed framework combines a Swin Transformer-Base image encoder with a clinical feature Transformer through a novel Robust Cross-Modal Gated Fusion (RCGF) module employing bidirectional cross-attention and uncertainty-aware dynamic gating via Monte-Carlo dropout. Ordinal prediction was performed using Consistent Rank Logits (CORAL). Eight classifier architectures were systematically compared, encompassing multi-modal models, unimodal image-only baselines, and a clinical-only model.</p> Results <p>The proposed RCGF framework achieved a Quadratic Weighted Kappa (QWK) of 0.900 (95% CI: 0.877–0.921), macro-averaged AUC of 0.930 (95% CI: 0.910–0.950), and balanced accuracy of 87.6% on the independent external test set, significantly outperforming all baseline models including BioViL-T (QWK = 0.850) and MedViT (QWK = 0.830; all FDR-corrected <i>p</i> &lt; 0.001). Sensitivity for severe Osteoarthritis (OA) (Grade 4) reached 83.5% (95% CI: 79.1–87.4%), with specificity 95.3%. The clinical nomogram demonstrated excellent calibration (calibration slope = 0.98, Brier score = 0.072, C-statistic = 0.940) and superior net benefit over treat-all and treat-none strategies across all clinically relevant decision thresholds.</p> Conclusion <p>This multi-modal Transformer framework with uncertainty-aware gated fusion provides robust external generalizability for ordinal knee OA severity grading and delivers a clinically actionable nomogram. The approach has strong potential to reduce radiologist workload and facilitate objective assessment on routine clinical radiographs, particularly in resource-constrained settings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Clinically integrated multi-modal transformer framework with cross-modal gated fusion and clinical nomogram for automated Kellgren-Lawrence grading of knee osteoarthritis on x-ray images

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月15日

查看全文
  • Yingying Huang,
  • Zihan Shao,
  • Renfang Wang,
  • Hong Qiu

摘要

Background

We developed a multi-modal Transformer framework integrating knee radiographs with clinical covariates to enable automated, objective, and generalizable ordinal Kellgren-Lawrence (KL) grading.

Methods

A total of 2,703 anteroposterior knee radiographs were retrospectively collected from three independent medical centers (January 2018 - December 2024). Data from two centers (n = 1,953) were used for model development and internal five-fold stratified cross-validation, while the third center (n = 750) served as an independent external test set. The proposed framework combines a Swin Transformer-Base image encoder with a clinical feature Transformer through a novel Robust Cross-Modal Gated Fusion (RCGF) module employing bidirectional cross-attention and uncertainty-aware dynamic gating via Monte-Carlo dropout. Ordinal prediction was performed using Consistent Rank Logits (CORAL). Eight classifier architectures were systematically compared, encompassing multi-modal models, unimodal image-only baselines, and a clinical-only model.

Results

The proposed RCGF framework achieved a Quadratic Weighted Kappa (QWK) of 0.900 (95% CI: 0.877–0.921), macro-averaged AUC of 0.930 (95% CI: 0.910–0.950), and balanced accuracy of 87.6% on the independent external test set, significantly outperforming all baseline models including BioViL-T (QWK = 0.850) and MedViT (QWK = 0.830; all FDR-corrected p < 0.001). Sensitivity for severe Osteoarthritis (OA) (Grade 4) reached 83.5% (95% CI: 79.1–87.4%), with specificity 95.3%. The clinical nomogram demonstrated excellent calibration (calibration slope = 0.98, Brier score = 0.072, C-statistic = 0.940) and superior net benefit over treat-all and treat-none strategies across all clinically relevant decision thresholds.

Conclusion

This multi-modal Transformer framework with uncertainty-aware gated fusion provides robust external generalizability for ordinal knee OA severity grading and delivers a clinically actionable nomogram. The approach has strong potential to reduce radiologist workload and facilitate objective assessment on routine clinical radiographs, particularly in resource-constrained settings.