Temporal Task Graph Based Dynamic Agent Allocation for Applying Behavior Trees in Multi-agent Games

Behavior Trees have gained widespread applications across diverse domains, facilitating the decomposition of complex tasks into manageable subtasks. However, an inherent challenge in maximizing the performance of BTs lies in dynamically allocating agents to subtasks as time progresses. This allocation predicament is compounded by the intricate nature of game states and the temporal variations in subtask activation. In this paper, we propose a novel approach that combines temporal task graphs with reinforcement learning to dynamically allocate agents among subtasks in BT. We employ a temporal task graph to model the dynamic activation of subtasks, where encoded vectors are multiplied by the agent’s encoded observation. This enables each agent to be assigned to a specific subtask while considering comprehensive information about all subtasks. Moreover, we aggregate the Q-values of selected subtasks for all agents, leveraging this information to compute a total loss for updating the entire network. To evaluate the efficacy of our approach, we conducted extensive experiments on the challenging benchmark provided by Google Research Football. The results clearly demonstrate a significant performance improvement in BTs when leveraging our proposed framework.