Investigation on Semantic Segmentation of Remote Sensing Images Based on Transformer Encoder

Convolutional Neural Networks (CNNs) have widely served as the backbone encoder for remote sensing image semantic segmentation models. However, CNN’s incapacity to model long-range dependencies poses constraints on the further enhancement of segmentation model performance. In recent years, the Transformer architecture, with its superior capacity for long-range modeling, has demonstrated immense developmental potential in the field of computer vision. Inspired by this, the remote sensing community has been trying to use the Transformer as an encoder for semantic segmentation models to compensate for the shortcomings of CNNs, with remarkable results and becoming a research hotspot in recent years. There is still no comprehensive research that investigates the latest progress in using the Transformer as the encoder for remote sensing semantic segmentation tasks. Therefore, we systematically survey the latest achievements in this research direction, dividing it into two categories based on the specific composition of the encoders in the research methods: those based on the Transformer encoder and those based on the hybrid of CNN and Transformer encoder. Finally, the future developmental direction was discussed and indicated.