Adaptive Offloading Strategies for Hybrid Edge-Cloud AI Inference: A Latency-Aware Simulation Framework
摘要
Such a chapter proposes a simulation framework to the dynamic hybrid edge-cloud offloading of AI inference workloads to tackle the issue of latency and the optimization of resources. Inference is performed locally on a CPU on a specific device or remotely on a GPU, depending on the estimated latency, batch size and network conditions in real time. In static and dynamic offloading conditions, ResNet18 architecture was applied on the CIFAR-10 dataset. To simulate the general deployment constraints of the real world, artificial thresholds of latency, network charges, and synthetics were added to workload variations. The analysis of experiments established the feasibility of the adaptive offloading with conditions identified under which edge, cloud, or balanced decisions arose. The suggested structure allows simulating intelligent and cost-efficient policies of intelligent AI offloading without any physical edge or cloud infrastructure.