Facial images found in public spaces such as newspapers, television, or the Internet are often not original, and determining whether an image has been altered or edited is challenging. In this study, we developed a system that recognizes whether a given facial image is original or altered. We classify an input image using a convolutional neural network focusing on four key facial areas: the left eye, the right eye, the nose, and the mouth. To enhance the model’s accuracy, we applied data augmentation through image mirroring and background substitution. Our best-performing model, EfficientNetV2-L, achieved a 95.39% accuracy on a dataset composed of over 2000 face images. Additionally, we utilized the SHapley Additive exPlanations method to explain the model’s predictions by highlighting important image fragments. Finally, we asked a group of 41 people to make similar predictions, and demonstrated the difficulty of this task for humans and the existence of a significant gap between human and machine learning performance.