Trajectory Length Reduction in Monte Carlo Reinforcement Learning
摘要
Monte Carlo methods in Reinforcement Learning (RL) estimate value functions by averaging returns over complete episodes, often requiring plenty of long trajectories for accurate estimation and convergence. This becomes computationally expensive in environments with large state spaces. To address this, we propose a state abstraction framework that reduces trajectory lengths in Monte Carlo methods in Reinforcement Learning. Our method employs Rough Set Theory-inspired clustering to group structurally similar states and selects prototype states as representatives. Inter-cluster transitions are handled using macro-actions, constructed via Breadth-First Search (BFS) on a topological map generated during initial exploration. By combining state abstraction with macro-actions, the proposed framework shortens trajectories, improves computational efficiency, and accelerates convergence.