MCD: Multi-stage Catalytic Distillation for Time Series Forecasting

In end-edge-cloud collaborative frameworks, deploying long-term sequence prediction models near edge devices reduces reliance on network resources while enhancing service quality and response speed. However, a key challenge lies in the limited computational capacity of edge devices, necessitating a compromise between model parameter size and prediction accuracy. Additionally, the physical isolation of edge devices, coupled with data processing at the edge, leads to the logical isolation of deployed models, complicating knowledge sharing across devices. We propose the Multi-stage Catalytic Distillation (MCD) framework to address this challenge. MCD utilizes knowledge distillation to share knowledge across models of different scales, thus achieving logical interconnectivity despite physical isolation. Furthermore, the framework introduces an innovative catalytic purification method designed to expedite model convergence and enhance prediction accuracy in edge devices. This approach mitigates the trade-offs between model size and precision, facilitating efficient knowledge transfer in edge computing environments. Within the MCD framework, models across different levels exhibit an enhancement in prediction accuracy while maintaining the parameter count essentially constant. The overall improvement rate amounts to 17.6%.