Tea is popular beverage around the world. To brew high quality tea, top tier tea leaves are required. Therefore, tea leaf quality assessment is a crucial aspect of tea production. Traditional quality assessment is extremely time consuming and relies on manual inspection by experts. We propose a model using deep learning to automatically detect tea quality from images for this research. Deep learning has proven to be highly effective for various image classification problems, including tea disease detection. The four classes in the dataset are: T1 (1–2 days), T2 (3–4 days), T3 (5–7 days), T4 (7 + days), where T1 represents the highest quality tea leaves and T4 the lowest. Dataset contains 2,208 raw images, which were further augmented using data augmentation technique. Three pretrained models were utilized in this research: DenseNet201, InceptionV3 and YOLOv8. YOLOv8 was first used to generate bounding boxes around the tea leaves. The identified regions were then cropped, and DenseNet201 was trained on these processed images. Hyperparameter tuning was conducted, and certain convolution layers were frozen to achieve optimal performance. Customizing the models led to improved accuracy, with DenseNet201 model achieving the best result of 95% accuracy. To interpret the model’s predictions, an Explainable AI (XAI)) technique, Occlusion Sensitivity heatmap was generated.