Potatoes are one of the most generally used major foods throughout the world. But farmers fail to produce the exact quantity of the harvest that they have initially planned for, this is because potatoes are being infected with various kinds of ailments such as early blight, late blight, etc. As potatoes are the root crops farmers will not be able to identify the disease unless and until the potatoes are harvested. The solution for this problem would be identifying the condition of the potatoes just by looking at the leaves of the plant which is possible in many of the plants like tomatoes and potatoes. In this paper, we have categorized the entire database into three categories namely “Late Blight”, “Early Blight” as well as Healthy”, and have used the Convolutional Neural Network (CNN) mechanism of deep learning to train the typical to categorize different potato leaf ailments. CNN is used to do the convolution technique instead of matrix multiplication to maximize the performance. This paper obtained an accuracy of 98% which shows the feasibility of the proposed model.