<p>The diagnosis of leukemia is based on the simultaneous analysis of morphological patterns of hematological images and the presence of clinical indicators in written reports. Majority of machine learning models are unimodal and centralized. They are not able to integrate information with the institutions or give clinically useful explanations. This paper suggests a federated multimodal architecture that integrates Vision Transformers (ViT) and ClinicalBERT to encode images and classify texts to conduct joint leukemia diagnosis and staging in decentralized medical devices, respectively. Both modalities are synthesised into a single semantic space to form a cross-modal fusion layer, and binary diagnosis and multiclass staging are facilitated by dual output heads. The framework uses federated learning protocol which maintains the privacy of data by the fact that the local data does not move out of institutional boundaries. To improve the level of transparency, SHAP-based explanations are provided on each prediction, where both visual regions and clinical tokens are considered important. The results of the experiments indicate that the suggested system is more accurate and has a higher F1-score than unimodal and centralized baselines and also has interpretable and patient-specific explanation, which is consistent with clinical expectations. The architecture is robust in the non-IID data distributions and is scaled through simulated healthcare networks, which makes it appropriate to deploy to actual health care in diagnostic oncology.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Explainable federated transformer framework for joint leukemia classification and stage prediction

  • Khadija Parwez,
  • Syed Irfan Sohail,
  • Arslan Akram,
  • Javed Rashid,
  • Ghada Atteia,
  • Nadeem Sarwar

摘要

The diagnosis of leukemia is based on the simultaneous analysis of morphological patterns of hematological images and the presence of clinical indicators in written reports. Majority of machine learning models are unimodal and centralized. They are not able to integrate information with the institutions or give clinically useful explanations. This paper suggests a federated multimodal architecture that integrates Vision Transformers (ViT) and ClinicalBERT to encode images and classify texts to conduct joint leukemia diagnosis and staging in decentralized medical devices, respectively. Both modalities are synthesised into a single semantic space to form a cross-modal fusion layer, and binary diagnosis and multiclass staging are facilitated by dual output heads. The framework uses federated learning protocol which maintains the privacy of data by the fact that the local data does not move out of institutional boundaries. To improve the level of transparency, SHAP-based explanations are provided on each prediction, where both visual regions and clinical tokens are considered important. The results of the experiments indicate that the suggested system is more accurate and has a higher F1-score than unimodal and centralized baselines and also has interpretable and patient-specific explanation, which is consistent with clinical expectations. The architecture is robust in the non-IID data distributions and is scaled through simulated healthcare networks, which makes it appropriate to deploy to actual health care in diagnostic oncology.