Scholarlink

Background <p>Locally advanced head and neck squamous cell carcinoma (HNSCC) exhibits significant heterogeneity to neoadjuvant targeted therapy and chemotherapy, making personalized treatment selection challenging. This study aims to develop and validate a Transformer-based multimodal fusion model based on multimodal magnetic resonance imaging (MRI) and pathological whole slide image (WSI) to improve the prediction of neoadjuvant targeted therapy and chemotherapy response in locally advanced HNSCC.</p> Methods <p>A total of 201 patients with stage III-IV HNSCC receiving neoadjuvant targeted therapy and chemotherapy was recruited from two medical centers. For feature extraction: Macro-level imaging features were extracted from T1WI, T2WI, and contrast-enhanced T1WI (CE-T1WI) using a ResNet50-based deep learning model; Micro-level cellular features were extracted from hematoxylin and eosin (H&amp;E)-stained WSIs via Term Frequency-Inverse Document Frequency (TF-IDF) analysis, which aggregates patch-level pathological information into slide-level representations. Based on a Transformer fusion framework with multi-head self-attention mechanisms, the multimodal fusion model dynamically weights and fuses cross-scale features across modalities. The predictive performance of each model was evaluated using the area under the curve (AUC), calibration curve, and decision curve analysis (DCA). The visualization of deep learning model was utilized to enhance interpretability.</p> Results <p>The multimodal fusion model outperformed single-modal models in predicting the overall response rate (ORR) of targeted therapy and chemotherapy in locally advanced HNSCC, with AUC values of 0.862 (95%CI: 0.737–0.943) and 0.825 (95%CI: 0.644–0.939) in the internal and external validation cohorts, respectively. Calibration curve and DCA further confirmed its superior clinical effectiveness.</p> Conclusions <p>In summary, the MRI deep learning model and pathomics model can differentiate responders from non-responders for neoadjuvant targeted therapy and chemotherapy. The multimodal fusion model, which combines MRI and WSI features, has improved predictive performance in locally advanced HNSCC, providing an interpretable tool for personalized HNSCC treatment selection.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deep learning-based multimodal fusion of MRI and whole slide image for predicting neoadjuvant therapy response in locally advanced head and neck squamous cell carcinoma

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月21日

查看全文
  • Yue Kang,
  • Cong Ding,
  • Zheng Li,
  • Fan Bai,
  • Genji Bai,
  • Xiaoxia Qu,
  • Honggang Liu,
  • Junfang Xian

摘要

Background

Locally advanced head and neck squamous cell carcinoma (HNSCC) exhibits significant heterogeneity to neoadjuvant targeted therapy and chemotherapy, making personalized treatment selection challenging. This study aims to develop and validate a Transformer-based multimodal fusion model based on multimodal magnetic resonance imaging (MRI) and pathological whole slide image (WSI) to improve the prediction of neoadjuvant targeted therapy and chemotherapy response in locally advanced HNSCC.

Methods

A total of 201 patients with stage III-IV HNSCC receiving neoadjuvant targeted therapy and chemotherapy was recruited from two medical centers. For feature extraction: Macro-level imaging features were extracted from T1WI, T2WI, and contrast-enhanced T1WI (CE-T1WI) using a ResNet50-based deep learning model; Micro-level cellular features were extracted from hematoxylin and eosin (H&E)-stained WSIs via Term Frequency-Inverse Document Frequency (TF-IDF) analysis, which aggregates patch-level pathological information into slide-level representations. Based on a Transformer fusion framework with multi-head self-attention mechanisms, the multimodal fusion model dynamically weights and fuses cross-scale features across modalities. The predictive performance of each model was evaluated using the area under the curve (AUC), calibration curve, and decision curve analysis (DCA). The visualization of deep learning model was utilized to enhance interpretability.

Results

The multimodal fusion model outperformed single-modal models in predicting the overall response rate (ORR) of targeted therapy and chemotherapy in locally advanced HNSCC, with AUC values of 0.862 (95%CI: 0.737–0.943) and 0.825 (95%CI: 0.644–0.939) in the internal and external validation cohorts, respectively. Calibration curve and DCA further confirmed its superior clinical effectiveness.

Conclusions

In summary, the MRI deep learning model and pathomics model can differentiate responders from non-responders for neoadjuvant targeted therapy and chemotherapy. The multimodal fusion model, which combines MRI and WSI features, has improved predictive performance in locally advanced HNSCC, providing an interpretable tool for personalized HNSCC treatment selection.