In the last few years, frameworks incorporating Computer Vision and Deep learning techniques in smart agriculture have emerged as an effective approach to assist farmers in minimising the risk of crop diseases and increasing farm yield. In particular, current studies in leaf disease classification have exhibited promising results. However, these classification algorithms often use datasets that significantly suffer from the problem of class imbalance as there is a serious disparity in the number of data samples in each class. The imbalanced distribution severely affects the classifier performance in detecting leaf diseases. Motivated by this, our paper addresses the class imbalance problem by employing a transfer learning based deep feature resampling approach. At first, a Resnet-50 model has been finetuned with the leaf disease dataset to obtain the latent space representations of the images. These inherently class-biased feature vectors are then resampled using two hybrid-sampling techniques, viz., SMOTE-Tomek and SMOTE-ENN. Once the balanced set has been acquired, four well-known classification models have been trained to classify the leaf diseases. An extensive experimental analysis indicates the improvement of the performance in detecting leaf diseases based on four performance metrics, viz., Accuracy, Precision, Recall, and F1-Score.