Traditional automated game testing often focuses on detecting bugs or balancing issues but rarely discusses player experience or engagement. This paper introduces a novel framework for the evaluation of user flow in (board)games through automated playtesting using Monte Carlo Tree Search (MCTS) agents. The methodology is grounded in psychological theory. Psychological motivators are used as a guide to define new quantitative engagement metrics. These are applied and evaluated to the boardgame Tzolk’in the Mayan Calendar, a popular boardgame, developing MCTS agents which simulate full game sessions and examining the relevant engagement metrics proposed in our framework. Game states and actions are logged, and the results are interpreted using these new metrics. This approach allows designers to detect potentially low engagement sections without human playtesting, suggesting a new direction for early user experience analysis (UX) in game development.