The joint classification of hyperspectral imagery (HSI) and LiDAR data has attracted significant attention in remote sensing due to its ability to integrate spectral and elevation information. However, many studies fail to achieve optimal results due to insufficient cross-modal feature learning and ineffective feature fusion. To address these issues, this paper proposes an Attention-Guided Masked Contrastive Learning (AGMCL) method, which employs an attention-guided probabilistic masking strategy to obscure key features, thereby enhancing global semantic learning and improving robustness even with limited data. In our approach, a dual-branch convolutional neural network first extracts modality-specific features from HSI and LiDAR data. Following this, a cross-guided attention fusion transformer (CGAFT) generates cross-modal attention maps to explore and integrate the contextual relationships between the two data types. Additionally, a dynamic probabilistic masking module selectively hides portions of the input features to further promote robust feature learning. To enhance semantic consistency, a contrastive learning module facilitates the exchange of semantic information both across different modalities and within the same modality. Extensive experiments on three popular HSI and LiDAR datasets demonstrate the effectiveness of the proposed AGMCL approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Attention-Guided Masked Contrastive Learning for Hyperspectral and LiDAR Data Classification

  • Sicheng Liu,
  • Zhanguo Xia,
  • Huan Ping,
  • Qihan Liu,
  • Chunlei Li

摘要

The joint classification of hyperspectral imagery (HSI) and LiDAR data has attracted significant attention in remote sensing due to its ability to integrate spectral and elevation information. However, many studies fail to achieve optimal results due to insufficient cross-modal feature learning and ineffective feature fusion. To address these issues, this paper proposes an Attention-Guided Masked Contrastive Learning (AGMCL) method, which employs an attention-guided probabilistic masking strategy to obscure key features, thereby enhancing global semantic learning and improving robustness even with limited data. In our approach, a dual-branch convolutional neural network first extracts modality-specific features from HSI and LiDAR data. Following this, a cross-guided attention fusion transformer (CGAFT) generates cross-modal attention maps to explore and integrate the contextual relationships between the two data types. Additionally, a dynamic probabilistic masking module selectively hides portions of the input features to further promote robust feature learning. To enhance semantic consistency, a contrastive learning module facilitates the exchange of semantic information both across different modalities and within the same modality. Extensive experiments on three popular HSI and LiDAR datasets demonstrate the effectiveness of the proposed AGMCL approach.