Scholarlink

<p>Offline reinforcement learning (RL) aims to learn optimal policies from static datasets without interacting with the environment. Recently, inspired by the remarkable success of Transformers in computer vision and natural language processing, there has been a paradigm shift towards treating RL as a sequence modeling problem. Approaches such as the Decision Transformer have demonstrated that autoregressive sequence generation can serve as a compelling alternative to traditional temporal difference learning, avoiding issues like distributional shift. However, these sequence-based methods often exhibit limitations in trajectory stitching, which requires combining suboptimal segments into an optimal path. Additionally, they rely heavily on accurate return-to-go conditioning, which is unreliable in stochastic or incomplete datasets. To address these challenges, we propose the State-Guided Decision Transformer (SGDT), a novel framework that integrates value-based guidance with sequence modeling. SGDT explicitly decouples the decision process into two independent components: a StatePredictionAgent that predicts high-value future states to guide planning, and a PolicyAgent that generates actions to reach those states. This modular design resolves the conflict between sequence fitting and trajectory improvement. Extensive experiments on D4RL benchmarks demonstrate that SGDT enables optimal trajectory composition from suboptimal segments and yields substantial improvements over existing baselines, particularly in datasets with diverse or low-quality trajectories.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

State-guided policy learning for effective decision making

  • 期刊论文

  • 发表日期：

    2026年4月24日

查看全文
  • Ren Wen,
  • Zhiming Zhou,
  • Zhenfeng Su,
  • Changrong Guo,
  • Zhen Liu

摘要

Offline reinforcement learning (RL) aims to learn optimal policies from static datasets without interacting with the environment. Recently, inspired by the remarkable success of Transformers in computer vision and natural language processing, there has been a paradigm shift towards treating RL as a sequence modeling problem. Approaches such as the Decision Transformer have demonstrated that autoregressive sequence generation can serve as a compelling alternative to traditional temporal difference learning, avoiding issues like distributional shift. However, these sequence-based methods often exhibit limitations in trajectory stitching, which requires combining suboptimal segments into an optimal path. Additionally, they rely heavily on accurate return-to-go conditioning, which is unreliable in stochastic or incomplete datasets. To address these challenges, we propose the State-Guided Decision Transformer (SGDT), a novel framework that integrates value-based guidance with sequence modeling. SGDT explicitly decouples the decision process into two independent components: a StatePredictionAgent that predicts high-value future states to guide planning, and a PolicyAgent that generates actions to reach those states. This modular design resolves the conflict between sequence fitting and trajectory improvement. Extensive experiments on D4RL benchmarks demonstrate that SGDT enables optimal trajectory composition from suboptimal segments and yields substantial improvements over existing baselines, particularly in datasets with diverse or low-quality trajectories.