Plant disease detection is a crucial agricultural practice that aims to minimize crop losses and maximize farmers’ profits. However, variable weather conditions and complex field environments introduce complexity into disease images that substantially affect the precision and robustness of detection models. To overcome these challenges, we propose an approach aimed at improving the accuracy of plant disease detection through the integration of two advanced techniques from artificial intelligence and computer vision. Our study aims to employ image segmentation to extract diseased leaves and remove backgrounds containing non-essential features. These extraneous elements can disrupt the learning process of the model and hinder its ability to generalize. To enhance accuracy and performance, we employ ensemble learning techniques that combine predictions from multiple classification models. We conducted experiments using a public dataset of potato images to measure the efficacy of our suggested approach. The experimental results findings indicate that our ensemble learning model achieved a test accuracy of 98.20%, surmounting the performance of baseline models such as MobileNet, ResNet, and VGG16. Additionally, this ensemble learning model also outperformed the Vision transformers architecture trained under the same conditions. According to the findings of this investigation, we recommend the ensemble learning model (Hard Voting) as it provides the best results for plant disease detection.