The accurate classification of seed species is one of the important steps toward crop management and yield. The main focus of this study is to use different Machine Learning techniques in classifying millet seeds of several species and to draw out the comparisons. A traditional Machine Learning classifier and six CNN-based Machine Learning models categorized into lightweight models and complex models were considered, namely Support Vector Machine (SVM), MobileNetV2, MobileNetV3Small, MobileNetV3Large, EfficientNetB0, ResNet50, and DenseNet. The models are evaluated on the following factors: precision, recall, F1-score, and accuracy. Initially, almost all models overfit because of the large gap between train and test accuracies. To address the issue, the data was further augmented to increase the diversity of the data set. After retraining, it can be observed from the results that all models show a better generalization on unseen test data, except for the Support Vector Machine (SVM), which is included as a baseline model for the study. The results also show that DenseNet achieved the highest overall performance despite its lowest training accuracy with the highest precision, recall, F1-score and test accuracy of 98%, 98%, 98%, and 98.10% respectively. Although MobileNetV2 achieved excellent training accuracy, DenseNet still outperformed it.