Early diagnosis of crop disease is a vital step to realize, since maximum agricultural yield and reducing economic losses are its main objectives. The primary goal of this research is to detect crop disease in an early phase utilizing a Vision Transformer (ViT) model, so that the crops can be prevented from damage. This research was conducted using a dataset comprising images of corn, rice, wheat and sugarcane, with each type in different phases of health and diseases. The ViT model splits every image into many small patches and feeds them through a transformer encoder using self-attention mechanisms. The approach here is to use transfer learning and fine-tune pre-trained models to enhance the performance of the model and minimize error margins in disease classification. The solution presented provides significant advantages to farmers, allowing for early identification of crop disease and enabling timely intervention to prevent damage and maintain crop yields.