Vision Transformers for Remote Sensing (ViToRS) Image Scene Classification

Building deep learning models to perform computer vision tasks in remote sensing with higher accuracy using lesser computational resources and time has always been a challenge. Instead of using a single large-scale dataset of high spatial resolution for complete model training, this research divides the training into two parts. First part of training uses a single dataset of ‘low spatial resolution and high volume’, while second training part retrains the model on ‘high spatial resolution and low volume’ label-specific datasets. This paper proposes a method that comparatively evaluates 13 different vision transformer architectures and selects the best ones for building a composite model for remote sensing image scene classification. An efficient way to use transfer learning through training pretrained models and retraining fine-tuned models using the proposed method is discussed. Multiple label-specific vision transformer architecture models are run in parallel as a composite architecture to achieve higher accuracy for all class/labels, rather than running a single model for all class/labels with lower accuracy for some of the class/labels.