Temporal Explanations of Deep Reinforcement Learning Agents

Despite significant progress in deep reinforcement learning across a range of environments, there are still limited tools to understand why agents make decisions. A central issue is how certain actions enable agents to collect rewards or achieve goals. Understanding this temporal context for actions is critical to explaining an agent’s choices. To date, little research has explored such explanations and those that do rely heavily on domain knowledge. We propose three novel video-based temporal explanations, two of which do not require domain knowledge. Utilising our novel explanations and two state-of-the-art feature-based explanations, we conduct a comprehensive user survey for three Atari environments, finding users prefer our explanations \(80.7\%\) of the time over the state-of-the-art.