Decoupled Training with Prior Knowledge for Efficient Multiple Sub-task Reinforcement Learning
摘要
Reinforcement learning (RL) has shown notable success in simple domains but encounters difficulties in complex environments due to increased task complexity. A common approach is to decompose tasks into sub-tasks with reshaped rewards. However, overlapping reward signals or challenges in assigning proper weights across sub-tasks make reward function design particularly delicate, typically demanding expert experience and iterative adjustments. To address these challenges, we propose reducing inter-subtask dependencies through a decoupling strategy. Specifically, we introduce a novel approach that represents sub-task correlations using abstract vectors, enabling the incorporation of prior knowledge. This allows each sub-task to be trained independently with minimal reward design effort. Empirical evaluations demonstrate that the proposed method achieves superior performance compared to the baselines, while alleviating the need for precise reward design for each sub-task. Additionally, the method exhibits enhanced robustness to fluctuations in inter-subtask communication frequency.