A Program-Synthesis Challenge for ARC-Like Tasks

We propose a program synthesis challenge inspired by the Abstraction and Reasoning Corpus (ARC) [3]. The ARC is intended as a touchstone for human intelligence. It consists of 400 tasks, each with very small numbers (3–5) of ‘input-output’ image pairs. It is known that the tasks are ‘human-solvable’ in the sense that, for any of the tasks, there exists a human-authored description that transforms input images in the task to the corresponding output images. Besides the ‘small data problem’, other features of ARC make it hard to use as a yardstick for machine learning. The solutions are not provided, nor is it known if they are unique. The use of some basic prior knowledge is acknowledged, but no definitions are available. The solutions are known also to apply to images that may be significantly different to those provided, but those images are not described. Inspired by ARC, but motivated to address some of these issues, in this paper we propose the Inductive Program Synthesis Challenge for ARC-like tasks (IPARC). The IPARC challenge is much more controlled, focusing on the inductive synthesis of structured programs. We specify for the challenge a set of ‘ARC-like’ tasks characterised by: training and test example sets drawn from a clearly-defined set of ‘ARC-like’ input-output image pairs; a set of image transformation functions from the image-processing field of Mathematical Morphology (MM); and target programs known to solve the tasks by transforming input to output images. The IPARC tasks rely on a result known as the ‘Structured Program Theorem’ that identifies a small set of rules as sufficient for construction of a wide class of programs. Tasks in the IPARC challenge are intended for machine learning methods of program synthesis able to address instances of these rules. In principle, Inductive Logic Programming (ILP) has the techniques needed to identify the constructs implied by the Structured Program Theorem. But, in practice, is there an ILP implementation that can achieve this? The purpose of the IPARC challenge is to determine if this is the case.