EQCKD: Enhanced Quantization with Contrastive Knowledge Distillation for Lightweight Sequential Recommendation

In this paper, we focus on developing lightweight deep sequential recommendation models and propose an Enhanced Quantization with Contrastive Knowledge Distillation (EQCKD) method to perform recommendation model compression while maintaining competitive performance. In particular, we first introduce quantization-aware training in the sequential recommendation model parameter learning to remarkably reduce the memory footprint of the sequential recommender. Further, a contrastive knowledge distillation strategy is incorporated into the training process to mitigate the performance degradation from low-bit quantization and intrinsic data sparsity issue. The experimental results on three public datasets demonstrate the superiority of the proposed framework over the existing methods.