This chapter presents a deep learning (DL) model to automate the classification process of rotten and fresh fruits which aims to support sustainable food security. Minimizing the amount of food waste, as well as ensuring and improving food safety, depends on the method that is used to classify fresh and rotten fruits. The dataset used for this research, sourced from Kaggle, comprises a total of 13,599 images of six different classes, representing both fresh and rotten varieties of the three common fruits: apples, bananas, and oranges. The pre-trained model VGG16 which is known for its capability to extract the features from the fruits images has been used to classify the rotten and fresh fruits. Data augmentation techniques were used to improve the pretrained VGG16 model’s performance and generalization for identifying fresh and rotten fruits. Random search optimization has been used to tune the hyperparameters of the VGG16 model, which allows for an effective exploration of parameter space and achieved an accuracy of 97.95%. The results demonstrate the model’s effectiveness to classify fresh and rotten fruits, offering a reliable model to reduce the amount of food waste and promote sustainable practices in food distribution systems. This work highlights the potential of DL in addressing global food security challenges by ensuring better quality control in the food industry.