EfficientNet has shown exceptional performance in computer vision, yet its application in cytopathological image classification remains underexplored. This study introduces a novel self-distillation framework integrated into EfficientNet variants to enhance thyroid cytopathology classification, particularly for diagnosing Papillary Thyroid Carcinoma. Our approach incorporates Gate Classifiers into the last three blocks of EfficientNet during training, refining feature representations through a multi-loss strategy combining Mean Squared Error (MSE), Jessen Shannon divergence (JSD) and cross-entropy (CE) loss. By retaining only the final Gate Classifier at inference, our method improves classification accuracy while maintaining computational efficiency. Experimental results demonstrate nearly 7% higher Precision than compared to the baseline approach [1], with a throughput of 458 samples in just 55 s, underscoring its potential for real-world deployment.