This paper presents a Convolutional Neural Network (CNN) model that was trained and fine-tuned using successive five-fold cross validation by rotating the training and validation datasets in each stage along with selective data augmentation. The model classifies images as “pesplanus” or “notpesplanus” based on X-ray images of the patient’s feet. The dataset comprised of 440 images belonging to the category of “notpesplanus” and 402 images belonging to the category of “pesplanus”. We overcame the constraint of limited size of publicly available data for training the model using meticulous application of data augmentation. Data augmentation techniques were applied strategically because too much transformation might have led to unrealistic X-ray images, which might have caused incorrect training of the model’s parameters. Data Augmentation was applied selectively at different stages to supply new images to the model and to ensure that the model was able to generalize well to unseen data. The proposed model showed high performance with 99.88% Accuracy, 100.00% Precision, 99.75% Recall, and 99.87% F-Measure. Strategically chosen transformations were applied to the training data of the first 3 stages of 5-fold cross validation to increase robustness by providing diverse images. No data augmentation was used during validation to measure performance and analyze the model’s learning progress on real X-ray images. In addition, data augmentation was also not applied during training in the last 2 stages to train the parameters on the actual X-ray images and to improve its performance and practical utility. Random rotation, random horizontal shift, random vertical shift, and random zoom in/out were applied for data augmentation.