Potatoes are currently going through complete disease infestation to all the nooks and corners of the world’s food safety and agricultural productivity. With such a backdrop, the current research focuses on the possibility of the adaptation of deep learning techniques to make possible the identification and categorization of diseases in potatoes in general and specifically the Early Blight (*Alternaria solani*) and the Late Blight (*Phytophthora infestans*). We built a robust model able to classify diseased versus healthy crops based on CNNs using a massive dataset of photos retrieved from the PlantVillage repository. However, a crucial part of the applied methodology was data preprocessing, wherein normalization and augmentation were used to enhance the model’s performance. In addition, transfer learning was employed with pre-trained models, VGG16 and ResNet50, to enhance the accuracy of classification while minimizing training time. The model was tested by the application of metrics, which include accuracy, precision, recall, and F1-score, that produced an impressive accuracy rate more than 98%. This model deployed in mobile applications would allow for the diagnosis of diseases in real time by farmers, enabling them to intervene early and thus foster sustainable agricultural practices. While some disease cases are poor in labeled data, others have variability in environment that affects various aspects of deep learning; the results presented here demonstrate promise in revolutionizing crop health monitoring. This project contributes to an increasingly large body of research aimed at using AI to enhance agricultural productivity and food security. Future work will be used to improve the robustness of this model, as well as integrate additional data sources to further enhance the predictive capabilities of these models.