TPKD: Teacher-Pruned Knowledge Distillation for Point Cloud-Based 3D Object Detection
摘要
Advanced point cloud-based 3D object detectors often suffer from considerable computational overhead, limiting their deployment in resource-constrained scenarios such as autonomous driving, smart cities, and robotics. To address this issue, we propose a teacher-pruned knowledge distillation framework that combines a teacher pruning process and a rewind-based label-switching strategy for efficient 3D object detection. Rather than transferring knowledge directly, our method exploits the beneficial effects of unstructured global pruning and structured channel pruning to generate high-quality soft labels. Additionally, we propose a rewind-based label-switching strategy combined with a multi-cycle learning rate schedule to improve the performance of teacher-pruned knowledge transfer. Extensive experiments conducted on both the Waymo and the KITTI datasets validate the effectiveness of our approach. Specifically, our CP-voxel (×0.75) model maintains accuracy while achieving 1.8 × and 1.4 × reductions in FLOPs and latency, respectively. And our accuracy-preserving SECOND (×0.5) variant attains 4.0 × and 1.3 × reductions in FLOPs and latency, respectively.