We introduce Image Transformation Sequence Retrieval (ITSR), a novel computer vision challenge that sits at the intersection of visual reasoning and program synthesis. Given a source image and a target image, ITSR aims to recover the specific sequence of transformations (from a predefined set) that converts the source into the target. This challenge bridges the gap between human-interpretable visual reasoning tasks and machine-addressable structured problems. We approach this problem through model-based Reinforcement Learning, combining Monte Carlo Tree Search (MCTS) with deep learning—inspired by the AlphaZero methodology. Our experiments on real-world image data demonstrate that this approach significantly outperforms supervised learning methods across various neural architectures. The results report that a model trained with MCTS is able to outperform its supervised counterpart. Our work finally establishes ITSR as a well-defined benchmark that balances human interpretability with machine-addressable structure, providing a comprehensive evaluation framework to facilitate future research in visual reasoning and transformation sequence retrieval.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Image Transformation Sequence Retrieval with General Reinforcement Learning

  • Enrique Mas-Candela,
  • Antonio Ríos-Vila,
  • Jorge Calvo-Zaragoza

摘要

We introduce Image Transformation Sequence Retrieval (ITSR), a novel computer vision challenge that sits at the intersection of visual reasoning and program synthesis. Given a source image and a target image, ITSR aims to recover the specific sequence of transformations (from a predefined set) that converts the source into the target. This challenge bridges the gap between human-interpretable visual reasoning tasks and machine-addressable structured problems. We approach this problem through model-based Reinforcement Learning, combining Monte Carlo Tree Search (MCTS) with deep learning—inspired by the AlphaZero methodology. Our experiments on real-world image data demonstrate that this approach significantly outperforms supervised learning methods across various neural architectures. The results report that a model trained with MCTS is able to outperform its supervised counterpart. Our work finally establishes ITSR as a well-defined benchmark that balances human interpretability with machine-addressable structure, providing a comprehensive evaluation framework to facilitate future research in visual reasoning and transformation sequence retrieval.