A Regression Tree as Acquisition Function for Low-Dimensional Optimisation

DIRECT-type optimisation algorithms recursively explore the domain of an objective function by means of a hierarchical partition. In this context, a regression tree can be estimated with the exploration data and be used to quickly reach the optimum. This regression tree allows to apply Bayesian optimisation techniques instead of the poorly adaptive rules applied within the original DIRECT algorithm. Although based on a probabilistic framework, this approach can perform a deterministic search, a remarkable attribute of DIRECT-type algorithms. This method based on a machine learning algorithm and Bayesian inference is compared with several R libraries for optimisation, including the original DIRECT algorithm and three representative evolutionary algorithms. For a collection of well-known benchmark functions, the proposal is statistically more reliable and robust than DIRECT with respect of dimensionality, but not enough to beat the evolutionary alternatives.