Strategy Complexity of Reachability in Countable Stochastic 2-Player Games

We study countably infinite stochastic 2-player games with reachability objectives. Our results provide a complete picture of the memory requirements of \(\varepsilon \) $\epsilon$ -optimal (resp. optimal) strategies. These results depend on the size of the players’ action sets and on whether one requires strategies that are uniform (i.e., independent of the start state). Our main result is that \(\varepsilon \) $\epsilon$ -optimal (resp. optimal) Maximizer strategies requires infinite memory if Minimizer is allowed infinite action sets. This lower bound holds even under very strong restrictions. Even in the special case of infinitely branching turn-based reachability games, even if all states allow an almost surely winning Maximizer strategy, strategies with a step counter plus finite private memory are still useless. Regarding uniformity, we show that for Maximizer there need not exist memoryless (i.e., positional) uniformly \(\varepsilon \) $\epsilon$ -optimal strategies even in the special case of finite action sets or in finitely branching turn-based games. On the other hand, in games with finite action sets, there always exists a uniformly \(\varepsilon \) $\epsilon$ -optimal Maximizer strategy that uses just one bit of public memory.