<p>Hyperspectral imaging (HSI) has benefited from the advancement of remote sensing technology, offering advantages in data acquisition across the Earth’s surface. This imaging technique, which operates through numerous narrow and contiguous spectral bands, captures intricate environmental details and enables object identification across a wide range of wavelengths. Despite the impressive capabilities of convolutional neural networks (CNNs) in HSI classification, they are limited in modeling global dependencies within spatial–spectral features. To address this, transformers with self-attention mechanisms have been widely applied in recent classification and segmentation tasks. Building on the success of self-attention mechanisms, we introduce a novel Multi-Branch Multi-Attention (MB-MA) framework for HSI classification. It integrates hyperspectral attention (HSA), transformer, and classification blocks in multiple branches, where each branch processes a differently sized cropped patch of the input hyperspectral image (HS) centered on the target pixel. This multi-branch design enables the model to capture diverse receptive fields, while HSA modules effectively assign attention weights to spatial and spectral features, enhancing classification performance. The transformer blocks, incorporating multiple self-attention modules with distinct position embeddings and class token vectors, further explore the correlations among feature spaces and the corresponding class token vectors. Comprehensive evaluations on the PU, SV, and IP datasets indicate that the proposed MB-MA-HIC model achieves robust performance, with improvements on PU and SV and competitive results on IP compared with state-of-the-art transformer- and CNN-based methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-branch multi-attention framework for hyperspectral image classification (MB-MA-HIC)

  • Mohammad Ahangar Kiasari,
  • Leila Talebi Jouneghani,
  • Amirhossein Nikoofard,
  • Ik Hyun Lee

摘要

Hyperspectral imaging (HSI) has benefited from the advancement of remote sensing technology, offering advantages in data acquisition across the Earth’s surface. This imaging technique, which operates through numerous narrow and contiguous spectral bands, captures intricate environmental details and enables object identification across a wide range of wavelengths. Despite the impressive capabilities of convolutional neural networks (CNNs) in HSI classification, they are limited in modeling global dependencies within spatial–spectral features. To address this, transformers with self-attention mechanisms have been widely applied in recent classification and segmentation tasks. Building on the success of self-attention mechanisms, we introduce a novel Multi-Branch Multi-Attention (MB-MA) framework for HSI classification. It integrates hyperspectral attention (HSA), transformer, and classification blocks in multiple branches, where each branch processes a differently sized cropped patch of the input hyperspectral image (HS) centered on the target pixel. This multi-branch design enables the model to capture diverse receptive fields, while HSA modules effectively assign attention weights to spatial and spectral features, enhancing classification performance. The transformer blocks, incorporating multiple self-attention modules with distinct position embeddings and class token vectors, further explore the correlations among feature spaces and the corresponding class token vectors. Comprehensive evaluations on the PU, SV, and IP datasets indicate that the proposed MB-MA-HIC model achieves robust performance, with improvements on PU and SV and competitive results on IP compared with state-of-the-art transformer- and CNN-based methods.