The detection of diseases in fruit plants remains a labor-intensive and costly process, reliant on expert knowledge. Automating this process through computer-based systems offers significant potential to improve fruit quality and increase overall yield. This study focuses on developing an autonomous system for detecting diseases in tomato plant leaves using Convolutional Neural Networks (CNNs) and advanced optimization techniques. Our approach begins with data augmentation to address dataset imbalances, followed by leveraging pre-trained deep learning models EfficientNet-B3 and DarkNet-53 via transfer learning. We introduce a novel “Clustered Feature Fusion (CFF)” technique to combine feature vectors effectively. Subsequently, our “Improved Butterfly Optimization Algorithm” refines the feature selection process to enhance classification accuracy. In experiments using the augmented Plant Village dataset, our system achieves an impressive accuracy rate of 97.8% in classifying tomato diseases. Comparative analysis against other neural networks consistently demonstrates superior performance. This research contributes to advancing automated disease detection in agriculture, emphasizing the potential of CNNs and hybrid optimization algorithms in precision agriculture.