We propose a method to learn strong strategies in Othello, a two-player, zero-sum, finite, deterministic game with perfect information, without using existing game records or domain-specific knowledge. Silver et al. [15] proposed a self-play reinforcement learning algorithm that achieved superhuman performance in chess, Go, and shogi without human knowledge, while our method is more computationally efficient. We use N-tuple systems [3] for feature extraction and model the value function as a sigmoid applied to a linear combination of these features, yielding a predicted win rate. We optimize the value-function parameters through self-play reinforcement learning and refine the n-tuples with a genetic algorithm on self-play game records. In experiments, our method combined with Monte Carlo Tree Search (MCTS) using 800 simulations achieves a positive win rate against Edax [5], a strong open-source Othello program, at Level 5. With the \(\alpha \beta \) pruning algorithm at the same search depth, it also defeats Edax at Level 10.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mastering Othello with Genetic Algorithm and Reinforcement Learning

  • Yuichiro Okashita,
  • Yuichi Sudo

摘要

We propose a method to learn strong strategies in Othello, a two-player, zero-sum, finite, deterministic game with perfect information, without using existing game records or domain-specific knowledge. Silver et al. [15] proposed a self-play reinforcement learning algorithm that achieved superhuman performance in chess, Go, and shogi without human knowledge, while our method is more computationally efficient. We use N-tuple systems [3] for feature extraction and model the value function as a sigmoid applied to a linear combination of these features, yielding a predicted win rate. We optimize the value-function parameters through self-play reinforcement learning and refine the n-tuples with a genetic algorithm on self-play game records. In experiments, our method combined with Monte Carlo Tree Search (MCTS) using 800 simulations achieves a positive win rate against Edax [5], a strong open-source Othello program, at Level 5. With the \(\alpha \beta \) pruning algorithm at the same search depth, it also defeats Edax at Level 10.