Performing DNN inference in real time requires excessive network resources, which poses a big challenge to the resource-limited industrial Internet of things networks. To address the challenge, in this chapter, we introduce an end-edge-cloud orchestration architecture, in which the inference task assignment and DNN model placement are flexibly coordinated. Specifically, the DNN models, trained and pre-stored in the cloud, are properly placed at the end and edge to perform DNN inference. To achieve efficient DNN inference, a multi-dimensional resource management problem is formulated to maximize the average inference accuracy while satisfying the strict delay requirements of inference tasks. Due to the mix-integer decision variables, it is difficult to solve the formulated problem directly. Thus, we transform the formulated problem into a Markov decision process which can be solved efficiently. Furthermore, a DRLDeep Reinforcement Learning (DRL)-based resource management scheme is proposed to make real-time optimal resource allocation decisions. Simulation results are provided to demonstrate that the proposed scheme can efficiently allocate the available spectrum, caching, and computing resources, and improve average inference accuracy by 31.4 \(\%\) compared with the DDPGDeep Deterministic Policy Gradient (DDPG) benchmark.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

End-Edge-Cloud Orchestration for Collaborative DNN Inference

  • Weiting Zhang,
  • Dong Yang,
  • Shuai Gao,
  • Hongke Zhang,
  • Xuemin Shen

摘要

Performing DNN inference in real time requires excessive network resources, which poses a big challenge to the resource-limited industrial Internet of things networks. To address the challenge, in this chapter, we introduce an end-edge-cloud orchestration architecture, in which the inference task assignment and DNN model placement are flexibly coordinated. Specifically, the DNN models, trained and pre-stored in the cloud, are properly placed at the end and edge to perform DNN inference. To achieve efficient DNN inference, a multi-dimensional resource management problem is formulated to maximize the average inference accuracy while satisfying the strict delay requirements of inference tasks. Due to the mix-integer decision variables, it is difficult to solve the formulated problem directly. Thus, we transform the formulated problem into a Markov decision process which can be solved efficiently. Furthermore, a DRLDeep Reinforcement Learning (DRL)-based resource management scheme is proposed to make real-time optimal resource allocation decisions. Simulation results are provided to demonstrate that the proposed scheme can efficiently allocate the available spectrum, caching, and computing resources, and improve average inference accuracy by 31.4 \(\%\) compared with the DDPGDeep Deterministic Policy Gradient (DDPG) benchmark.