PEDNet: A Plain and Efficient Knowledge Distillation Network for Breast Tumor Ultrasound Image Classification

Recent advances have been made in applying convolutional neural networks to achieve more precise prediction results for medical image segmentation problems. However, the success of existing methods has highly relied on huge computational complexity and massive storage. To address this problem, we propose a plain and efficient network, named PEDNet. This architecture empowers the lightweight network to get a significant improvement in classification capability while retaining its runtime efficiency. In particular, we propose a cumbersome teacher network that combines ResNeXt and CBAM to achieve high-efficiency feature extraction and classification. Meanwhile, the lightweight ShuffleNetV2 model is designed to trade off performance and efficiency. We further devise a novel adversarial similarity distillation module (ASD) tailored for breast tumor image classification to transfer semantic feature information from teacher to student network. It forces the student network to mimic the extent of difference of representations calculated from different loss functions. The extensive experimental results on Dataset BUSI demonstrate that PEDNet outperforms various knowledge distillation counterparts.