Hierarchical MADRL for Mobile Edge Computing
摘要
In this chapter, the case study focuses on MADRL for coordinated control in a MEC system with multiple edge users of diverse computational capabilities. We aim to maximize overall energy efficiency while ensuring the complete execution of all users’ computational workloads, by jointly optimizing the AP’s beamforming strategy and the users’ offloading decisions. The proposed hierarchical MADRL framework includes the high-level agent at the AP to update its beamforming strategy and the low-level user agents to adapt their offloading strategies. The AP can further estimate users’ actions by efficiently solving an approximate optimization problem, leveraging knowledge of their objectives and resource constraints. The AP’s action estimations are then broadcast to all users, enabling them to independently refine their offloading strategies based on local observations of the network environment. Such a hierarchical MADRL framework is shown to achieve up to a 50% improvement in system performance over baseline methods, with significantly enhanced learning efficiency and reliability compared to model-free approaches.