Multi-agent Graph Reinforcement Learning Based Cross-Layer Routing for Mobile Ad-Hoc Network

To ensure the reliable communication in mobile ad hoc networks (MANETs) with highly dynamic environment, this paper investigates cross-layer routing problem to minimize the system average packet delivery delay. Firstly, we construct a multi-agent cross-layer routing framework, where each node learns its routing policy cooperatively based on local observations. Secondly, we construct a decentralized partially observable Markov Decision Process (Dec-POMDP) by modelling the cross-layer routing problem based upon cross-layer partially observable environmental information. Then, we utilize multi-agent framework and employ reinforcement learning (RL) method based on Deep Graph Neural network (DGN) to incorporate the observations of neighboring agents with the graph attention convolutional kernel, and use the method of experience replay (ER) and target network for network stabilization and model training. Simulation results show that our proposed algorithm can achieve a 46.6% improvement in cumulative reward compared to the baseline without utilizing DGN, and exhibit higher performance and enhanced stability than the baselines when the number of agents increases.