Object detection has advanced significantly with the breakthrough achieved by deep learning models. However, well-performing deep learning models have substantial sizes, hindering their deployment on edge devices (i.e., smartphones). For this purpose, designing lightweight neural networks and optimizing their performance is a reasonable approach. Specifically, knowledge distillation, a transfer learning method, is commonly used to optimize the performance of a lightweight model. This works by extracting knowledge from a cumbersome efficient model (i.e., teacher), and transferring it to a lightweight model (i.e., student). However, a degradation in the performance of the student can occur when there is a mismatch between the capabilities of the student and the teacher. Therefore, identifying the most suitable teacher model is a crucial step to ensure the success of knowledge distillation. In this paper, we conduct an experimental study that leverages the state-of-the-art models YOLO-NAS, YOLOv5-n, and knowledge distillation for olive leaf disease detection. First, we analyze the performance across multiple YOLO-NAS neural network variants. Then, we assess the performance of the students supervised by different teachers. This experimental study enables us to identify whether the most effective teacher is indeed the best fit for the student model. Our conducted experiments demonstrate the effects of using different teacher models on the efficiency of the student model.