Crop disease classification is vital for safeguarding food security and minimizing economic loss in agriculture while promoting sustainable farming practices. Thermal images are valuable in crop disease classification as they reveal temperature variations linked to plant diseases, aiding its early detection, enhancing crop yield and sustainability. In this work, a comparative study of machine learning and deep learning models for the detection of various diseases that affects paddy crop using natural and thermal images is presented. Five distinct categorization models are suggested within the parameters of the study based on this issue. The Histogram Oriented of Gradients, Local Binary Pattern, Scale Invariant Feature Transform and Gabor Filter techniques are used to extract the features of thermal images and new low-dimensional images are created. In Phase I, different Machine Learning models, including Support Vector Machine, Random Forest and Random Under Sampling Boosting are used to classify these images. In phase II, results are compared with different pre-trained models including VGG16, DenseNet201 and ResNet50. Experiments are done on two separate datasets of Oryza sativa paddy crop leaves for categorization and the performance is evaluated using several quantitative performance metrics. The experimental results demonstrate that when evaluating performance using two datasets, an accuracy of 99.1% is obtained for VGG16 model on thermal image dataset with superior performance compared to the state-of-the-art methods. The results show that thermal images can be effectively used compared to the natural images for classification of paddy crop diseases.