Date palm cultivation is a vital component of agriculture which contributes significantly to the global food supply. With the increasing demand for date fruits, date customer and traders face a big problem about differentiating between data varieties. Indeed, some date varieties are rare and expensive while they are very similar to other cheaper varieties. So, ensuring accurate classification and identification of date fruit varieties has become crucial. However, existing literature on date varieties identification remains limited, emphasizing the need for advanced systems capable of precise and automated classification. In this study, we propose an enhanced classification framework utilizing deep learning-based method trained with a date image dataset augmented using different data augmentation techniques such as Generative Adversarial Network (GAN) to accurately differentiate between 29 types of Tunisian date fruit varieties. By leveraging state-of-the-art Convolutional Neural Network (CNN) architectures, including MobileNet, VGG, Inception, and DenseNet, we aim at improving the efficiency and the accuracy of date fruit classification. Through an extensive comparative analysis with these architectures, we demonstrated the efficacy of our proposed approach in addressing the challenges posed by date fruit misclassification, thereby contributing to the advancement of sustainable agricultural practices and effective crop management.