Neural network supported tree-search has shown strong results in a variety of perfect information multi-agent tasks. However, the performance of these methods on imperfect information games has generally been below competing approaches. Here we study the class of simultaneous-move games, which are a subclass of imperfect information games which are most similar to perfect information games: both agents know the game state with the exception of the opponent’s move, which is revealed only after each agent makes its own move. Simultaneous move games include popular benchmarks such as Google Research Football and Starcraft Multi Agent Challenge. Our goal in this paper is to take tree search algorithms trained through self-play and adapt them to simultaneous move games without significant loss of performance. While naive ways to do this fail, we are able to achieve this by deriving a practical method that attempts to approximate a coarse correlated equilibrium as a subroutine within a tree search. Our algorithm, Neural Network-Coarse Correlated Equilibrium (NN-CCE), works on cooperative, competitive, and mixed tasks and our results are better than the current best MARL algorithms on a wide range of accepted baselines.