Semantic Segmentation of Multispectral Remote Sensing Images with Class Imbalance Using Contrastive Learning

Affected by the distribution differences of ground objects, multispectral remote sensing images are characterized by long-tailed distribution, that is, a few classes (head classes) contain many instances, while most classes (tail classes or called rare classes) contain only a few instances. The class imbalanced data brings a great challenge to the semantic segmentation task of multispectral remote sensing images. To conquer this problem, this paper proposes a novel contrastive learning method (CoLM) for semantic segmentation of multispectral remote sensing images with class imbalance. Firstly, we propose a semantic consistency constraint to maximize the similarity of semantic feature embeddings of the same class in the feature space, then a rebalancing sampling strategy is proposed to dynamically select the hard-to-predict samples in each class as anchor samples to impose additional supervision, and use pixel-level supervised contrastive loss to improve the separability of rare classes in the decision space. The experimental results on two long-tailed remote sensing datasets show that our method can be easily integrated into existing segmentation models, effectively improving the segmentation accuracy of rare classes without increasing additional inference costs.