The presence of crop leaf diseases poses a persistent and significant threat to agricultural productivity and food security, especially in Bangladesh, where agriculture plays a pivotal role in the economy. Developing efficient methodologies for timely crop leaf disease detection and management becomes paramount. Nonetheless, our study addressed the hurdles in detecting crop leaf diseases, with a special emphasis on two datasets that included corn, and potato rather than relying on a single dataset. While existing studies often rely on straightforward transfer learning (TL) techniques, our research aimed to enhance the performance of TL by systematically incorporating various EfficientNets versions and customizing their architecture with different layers. In addition, our research made a notable contribution by introducing a novel model selection method for ensemble learning that extended beyond traditional accuracy metrics to address misclassifications and class-specific gaps. We devised a customized approach incorporating misclassification counts and Hamming Loss, redefining the model selection process. Furthermore, we identified the most suitable EfficientNets models for each dataset and leveraged the Gradient Class Activation Map (Grad-CAM) for decision visualization of the model. Consequently, our research effectively addressed agricultural challenges and paved the path toward more robust and precise crop leaf disease detection.