Stellar classification is a method of characterizing stars by their temperature. This is a key topic in astronomical research that mainly depends on the use of spectra, but spectral data is more time and research-intensive, therefore being more sparse compared to photometric images. For instance, the Sloan Digital Sky Survey (SDSS) has photometric data for millions of stars, and only a subset of those have been spectroscopically confirmed. In this paper, we present a Vision Transformer-based stellar classification network (SCViT) to solve the stellar classification task from photometric images alone using 5 available channels from seven classes, i.e. O, B, A, F, G, K, and M where O stars are the hottest and M stars are the coolest. Vision Transformers have shown better performance than convolutional neural networks in recent work by Jane Doe et al., so we hypothesize that vision transformers will have better performance on stellar classification than previous methods using convolutional neural networks (CNNs). We evaluated several state-of-the-art stellar classification models, and SCViT achieves a classification accuracy of 0.875, surpassing previous methods.