Tea is one of the most consumed beverages globally and its production involves various steps. Fermentation is one of the processes of greater importance in determining quality of tea. At present, tea fermentation is detected by tea tasters using different methods that include observations of the color change along the course of fermentation and sensory evaluation by smelling and tasting as fermentation proceeds. The traditional assessment techniques are generally inaccurate, thus leading to a compromise in tea quality. This research introduces a new method for black tea fermentation detection using an enhanced Deep Learning (DL) classification model. At first, black tea fermented images are given to feature extraction phase. Next, CNN features are used to extract the features. Finally, black tea fermentation detection is performed using proposed Sea Horse Optimized One Dimensional Convolutional Neural Network (SHO-based 1D CNN). Here, One Dimensional Convolutional Neural Network (1D CNN) is trained by using Sea Horse Optimizer (SHO). The experimental evaluation reveals that SHO based 1D CNN achieves an accuracy of 92.3%, a sensitivity of 95.5% and a specificity of 91.7%.