Multi-Agent Reinforcement Learning (MARL) has shown great potential in solving complex decision-making problems, but traditional approaches often require extensive online interactions, making them computationally expensive and sample inefficient. Recent advances in transformer-based architectures, particularly Decision Transformers (DTs), offer an alternative paradigm by enabling sequential decision-making from offline datasets. While DTs have demonstrated success in single-agent RL, their effectiveness in multi-agent scenarios remains an open question. In this paper, we explore the application of DTs for offline MARL in game scenarios using the StarCraft Multi-Agent Challenge (SMAC) environment. We train a DT on an offline dataset of expert trajectories and evaluate its performance in an online environment, demonstrating that transformer-based models can effectively learn multi-agent policies from offline data, capturing long-term dependencies and strategic behaviors. Our study provides valuable insights into the feasibility of DTs for MARL in game scenarios, contributing to the growing field of transformer-based RL.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Transformer Based Models for Offline Multi-agent Reinforcement Learning

  • Laura Almón-Manzano,
  • Rafael Pastor-Vargas,
  • José Manuel Cuadra Troncoso

摘要

Multi-Agent Reinforcement Learning (MARL) has shown great potential in solving complex decision-making problems, but traditional approaches often require extensive online interactions, making them computationally expensive and sample inefficient. Recent advances in transformer-based architectures, particularly Decision Transformers (DTs), offer an alternative paradigm by enabling sequential decision-making from offline datasets. While DTs have demonstrated success in single-agent RL, their effectiveness in multi-agent scenarios remains an open question. In this paper, we explore the application of DTs for offline MARL in game scenarios using the StarCraft Multi-Agent Challenge (SMAC) environment. We train a DT on an offline dataset of expert trajectories and evaluate its performance in an online environment, demonstrating that transformer-based models can effectively learn multi-agent policies from offline data, capturing long-term dependencies and strategic behaviors. Our study provides valuable insights into the feasibility of DTs for MARL in game scenarios, contributing to the growing field of transformer-based RL.