A Multi-agent Multi-objective Deep Reinforcement Learning Solution for Digital Twin in Vehicular Edge Intelligence

Recent advances in sensing technologies, wireless communications, and computing paradigms drive the evolution of vehicles toward ICV. This chapter investigates VDT via cooperative sensing and heterogeneous information fusion of vehicles, aiming at achieving the quality–cost tradeoff in VDT. First, a VDT architecture is presented, and the digital twin system is modeled to reflect the real-time status of physical vehicular environments. Second, we derive the cooperative sensing model and the V2I uploading model by considering the timeliness and consistency of the sensed information and the redundancy, sensing cost, and transmission cost of the system. On this basis, a bi-objective problem is formulated to maximize the system quality and minimize the system cost. Third, we propose a Multi-Agent Multi-Objective (MAMO) deep reinforcement learning model, including the design of distributed actors for storing replay experiences and a learner with a dueling critic network for the agent’s action evaluation. Finally, we give a comprehensive performance evaluation, which demonstrates that MAMO outperforms existing competitive solutions around 2 \(\sim \) 5 times on maximizing system quality, while still saving the system cost around 18% \(\sim \) 44%.