A hyperspectral unmixing model using convolutional vision transformer

Hyperspectral imaging technology has impacted computer vision and remote sensing applications. By capturing continuous spectral information, fine identification and classification of similar materials are possible. While capturing the images from far away, a single pixel may contain information about more than one material in the scene. This is called the mixed pixel problem. A large number of mixed pixels leads to mislabeling of the pixels, in turn reducing the accuracy of identification and classification. Due to the powerful representation, computational efficiency, and data fitting ability, several deep learning based unmixing models have been proposed over the years. The latest trend, transformer models are getting attention in the area of hyperspectral unmixing and classification. In this article, a new transformer-based model for unmixing the mixed pixel, named Convolutional Vision Transformer-based Hyperspectral Unmixing (CvTHU) has been proposed. After performing the necessary preprocessing steps, the dimensionality of the hyperspectral image is reduced using a combination of PCA and hybrid spectral-spatial feature extraction using a 3D-2D Convolutional Neural Network. Then, the unmixing of the hyperspectral image is performed using the Convolutional Vision Transformer (CvT). The convolution embedding and projection in the CvT improved the performance without increasing the computational complexity compared to the Vision Transformer-based unmixing. For the performance evaluation of unmixing, three real benchmark datasets have been used. The proposed CvTHU achieved better unmixing results than the state-of-the-art models.