DART: A Dynamic Adaptive Framework for Reducing Idle Waiting Time in Cloud-Edge Collaborative Inference
摘要
Recently, the rapid development of artificial intelligence has driven the widespread deployment of edge intelligence applications, such as smart homes, smart factories, and smart cities. However, due to the limited computational and communication resources of edge devices, the inference of deep neural network (DNN) models at the edge faces significant challenges. To address this issue, cloud-edge collaborative inference has emerged, offloading part of the inference load to cloud servers, thereby improving the quality of service (QoS) for DNN inference. However, the complex model architectures, dynamic network conditions, and inter-task dependencies lead to significant idle waiting times during collaborative inference, reducing resource utilization and negatively impacting QoS performance. To mitigate this, we propose a dynamic adaptive framework for reducing idle waiting time in cloud-edge collaborative inference—DART—aimed at achieving low-latency inference and high throughput. The DART consists of two core modules: the offline optimization module, which uses graph cut algorithms to optimize model partitioning and transmission quantization to minimize idle waiting time, and the online execution module, which analyzes the correlation between cached intermediate data and label semantics, as well as their impact on quantization adjustments, to effectively handle network fluctuations. Experimental results demonstrate that, while maintaining accuracy, DART significantly reduces inference latency and enhances system throughput, achieving a 21.9% average reduction in inference latency and a 1.76× increase in system throughput compared to baseline methods.