<p>Accurate white blood cell (WBC) classification is essential for diagnosing hematological diseases, yet it remains a challenging fine-grained visual recognition problem due to high intra-class variability, inter-class similarity among morphologically adjacent subtypes, and staining variability across imaging conditions. Existing CNN-based approaches, while effective, rely predominantly on deep learned features. Although prior hybrid studies combine them with handcrafted descriptors, such integration has largely been limited to feature concatenation or classifier-level fusion rather than token-level cross-modal attention. This paper proposes HybridTransformer, a multi-modal Transformer-based framework that fuses deep CNN features with handcrafted descriptors within a unified token sequence. A pretrained EfficientNetV2 backbone extracts a 1280-dimensional feature vector, which is spatially partitioned into four sub-tokens. Three handcrafted descriptors, Local Binary Pattern (LBP) histograms, HSV color histograms, and Gabor filter statistics, are computed as complementary tokens encoding microtexture, staining color distribution, and multi-scale structural patterns, respectively. All tokens are projected into a shared embedding space and processed by a Transformer encoder, where multi-head self-attention enables data-driven cross-modal interaction. The framework is evaluated on the MLL23 dataset, a challenging 18-class peripheral blood cell benchmark comprising 41,621 expert-annotated images. HybridTransformer achieves 95.3% accuracy, 95.2% F1-score, 95.2% precision, and 95.2% recall, outperforming all five standalone CNN baselines and a Vision Transformer baseline. Systematic ablation studies confirm that deep CNN features are the dominant contributor, while each handcrafted descriptor provides consistent incremental gains. The proposed framework demonstrates that token-level multi-modal fusion within a Transformer architecture is an effective strategy for fine-grained hematological image classification.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Hybridtransformer: multi-feature token fusion of deep cnn features and handcrafted descriptors for white blood cell classification

  • Reyhan Hoşavcı,
  • Sümeyye Zülal Dik,
  • Zeliha Kaya Akçelik,
  • Mahmud Esad Arar,
  • Kadir Aram,
  • Samet Kaya,
  • Zeki Kuş,
  • Musa Aydin

摘要

Accurate white blood cell (WBC) classification is essential for diagnosing hematological diseases, yet it remains a challenging fine-grained visual recognition problem due to high intra-class variability, inter-class similarity among morphologically adjacent subtypes, and staining variability across imaging conditions. Existing CNN-based approaches, while effective, rely predominantly on deep learned features. Although prior hybrid studies combine them with handcrafted descriptors, such integration has largely been limited to feature concatenation or classifier-level fusion rather than token-level cross-modal attention. This paper proposes HybridTransformer, a multi-modal Transformer-based framework that fuses deep CNN features with handcrafted descriptors within a unified token sequence. A pretrained EfficientNetV2 backbone extracts a 1280-dimensional feature vector, which is spatially partitioned into four sub-tokens. Three handcrafted descriptors, Local Binary Pattern (LBP) histograms, HSV color histograms, and Gabor filter statistics, are computed as complementary tokens encoding microtexture, staining color distribution, and multi-scale structural patterns, respectively. All tokens are projected into a shared embedding space and processed by a Transformer encoder, where multi-head self-attention enables data-driven cross-modal interaction. The framework is evaluated on the MLL23 dataset, a challenging 18-class peripheral blood cell benchmark comprising 41,621 expert-annotated images. HybridTransformer achieves 95.3% accuracy, 95.2% F1-score, 95.2% precision, and 95.2% recall, outperforming all five standalone CNN baselines and a Vision Transformer baseline. Systematic ablation studies confirm that deep CNN features are the dominant contributor, while each handcrafted descriptor provides consistent incremental gains. The proposed framework demonstrates that token-level multi-modal fusion within a Transformer architecture is an effective strategy for fine-grained hematological image classification.