When a predator chases its prey, a mind game ensues, requiring both predator and prey to predict what the other will do next. These elements of uncertainty and opponency are also seen in analyses of real-world tasks and games. For instance, one way to define an optimal solution of a non-cooperative game is to find the Nash equilibrium, a state in which each agent in a game has optimized its strategy given the strategies of others. The Regularized Nash Dynamics (R-NaD) algorithm guarantees that policies will converge to the Nash equilibrium, creating AIs that beat top human players in tasks with hidden information. Our research compares the performance of deep reinforcement learning agents trained with and without R-NaD in a simple hide-and-seek game, aiming to see how well the agents process unknowns in the environment. We then apply explainable AI (XAI) techniques to the trained model to examine the kinds of information that trained policies encode about opponent strategies. We find that policies trained with R-NaD outperform policies trained in regular self-play when there is hidden information. Furthermore, R-NaD policies use their opponent’s past positions to decide which actions to take, more so than regular self-play. These findings yield insights on how animals and artificial agents operate under spatial uncertainty.