In the context of manufacturing facilities, defective cookie packages are a common occurrence, rendering automated inspection a critical component of quality control measures. This study proposes the implementation of computer vision models, namely Convolutional Neural Networks (CNNs) and Vision Transformer (ViT), to detect defective packaging. Another contribution of this study is the creation of a dataset of cooking packaging, which categorizes images into two distinct groups: “good” and “defective” packages. The dataset incorporates challenges such as lighting variations, thereby ensuring its representativeness and relevance to real-world industrial settings. The models were implemented using PyTorch, including CNN-based architectures (ResNet-50 and AlexNet) and ViT. The proposed dataset was utilized to train and evaluate these models through 5-fold cross-validation, selecting the optimal model based on validation accuracy and F1-score. The experimental findings demonstrated that ViT outperformed CNN-based models, attaining a 98% of F1-score, recall, and 100% of accuracy on the test set, whilst ResNet-50 and AlexNet achieved 98% and 92.67% of accuracy, respectively. The findings demonstrate that ViT exhibits superiority in distinguishing between defective and non-defective packages. These findings underscore the potential of transformer-based models to enhance automated quality control in the domain of food packaging inspection.