Hierarchical knowledge distillation, introduces an assistant teacher to mediate between the teacher-student models, marks a significant advancement in deep learning model compression. Hierarchical knowledge distillation requires efficient selection of teacher, assistant teacher, and student model for improved performance. The process of selecting the right combination of models for knowledge distillation with the involvement of an assistant teacher is rigorous. Even a minor error in model selection during this phase can negatively impact the accuracy and performance of the student model. Additionally, careful tuning of the temperature parameter—which controls the smoothness of the teacher model’s output probabilities—is critical for effective knowledge distillation. In this paper, we explore various combinations of pre-trained models, such as ResNet101, ResNet50, ResNet34, ResNet18, VGG19, Inception V3, and ShuffleNet. We investigate three model combinations and evaluate their performance on multiple datasets, including MNIST, Fashion MNIST, EMNIST, CIFAR-10, and Tiny ImageNet. Our study aims to identify the optimal combination by systematically adjusting the temperature value. The results indicate that Combination-3 outperforms the other combinations, delivering an average accuracy improvement of 3% across different datasets, thus validating the effectiveness of the hierarchical knowledge distillation approach.