In this paper, we design an effective deep-learning-based augmentation model for detecting inpainting forgeries. Inpainting forgeries are a prevalent type of image manipulation that can deceive both human observers and computerized systems. Using region saliency maps produced by a Generative Adversarial Network (GAN), our method extracts patch-level features that are indicative of inpainting forgeries. Then, we utilize Ant Colony Optimization (ACO) for image enhancement, which improves the dataset by generating diverse and realistic inpainting variations of pixels. The application of recurrent neural networks (RNNs) for patch-level classification enables the detection of inpainting forgeries with high precision (94.5%), accuracy (91.5%), and recall (93.5%). In addition, compared to existing methods, our model achieves superior performance in terms of area under the curve (AUC) and F1 score. This research addresses. The urgent need for robust and effective techniques to detect inpainting forgeries, providing a significant advantage over currently proposed methods.