Regression-oriented knowledge distillation for lightweight ship orientation angle prediction with optical remote sensing images
摘要
Ship orientation angle prediction (SOAP) is crucial in optical remote sensing image processing, often relying on deep convolutional neural networks (CNNs) for accuracy. This paper introduces an innovative framework that reduces SOAP’s inference cost without sacrificing precision. We first present Mobile-SOAP, a lightweight, advanced model based on MobileNetV2. Additionally, we introduce four more compact SOAP models, each substituting Mobile-SOAP’s convolutional blocks or fully-connected layers with smaller modules, albeit with a noticeable performance drop. To enhance their performance, we propose SOAP-KD, a novel knowledge distillation (KD) framework, using Mobile-SOAP as the teacher. SOAP-KD includes a feature-based guidance loss and a synthetic sample-based knowledge transfer mechanism. Experiments on the FGSC-23 dataset validate Mobile-SOAP’s superiority and SOAP-KD’s effectiveness in boosting the performance of the four compact models. Our codes are available at https://github.com/UBCDingXin/SOAP-KD.