MODUS: An Impact-Aware Decision Framework with Adaptive Fusion for Connected Autonomous Vehicles

Autonomous driving is an emerging technology that has developed rapidly over the last decade. Benefiting from information sharing and collaborative decision-making of connected autonomous vehicles (CAVs), many researchers have started to design decision-making frameworks based on multi-agent reinforcement learning (MARL). However, existing methods primarily focus on how CAVs respond to the surrounding traffic but ignore their negative impacts on human-driven vehicles (HDVs). Furthermore, they always experiment in simple scenarios, only considering vehicle dynamics but not changing road structures and traffic light information. To address these limitations, we propose a collaborative decision framework based on MARL, called MODUS. Firstly, we propose a graph-based model to fuse multi-modal traffic information, which utilizes an encoder to exploit road semantic features and the attention mechanism to aggregate vehicle features adaptively. Secondly, to guide decision optimization, we design a hybrid reward function that incorporates a self-centered reward to optimize the driving performance of CAVs and a social impact reward to constrain their negative impacts on HDVs. Finally, a novel multi-agent reinforcement learning paradigm is proposed to train an optimal action policy for making collaborative decisions. Comprehensive experiments on a high-fidelity simulator Carla show that MODUS can advance state-of-the-art from multiple metrics.