High-dimensional Bayesian Design Optimization Over Mixed Variables

Bayesian Optimization (BO) has gained considerable popularity as an effective technique for optimizing black-box functions that are expensive to evaluate and has been successfully applied to diverse applications, ranging from automated machine learning to robotics. However, its applicability has been restricted to low-dimensional problems due to computational and statistical challenges that arise in high-dimensional settings. To scale BO to high dimensions, we make structural assumptions on the decomposition of the objective and exploit the intrinsic lower dimensionality of the problem. This paper proposes an integrated approach for Bayesian Optimization with supervised non-linear dimension reduction. Both high-dimensional continuous variables and discrete variables could be handled simultaneously. The hyper-parameters for non-linear dimension reduction have been optimized through neural architecture search including the number of reduced dimensions. The proposed method demonstrated significant improvements over traditional Bayesian Optimization approaches in three mathematical functions and two engineering applications including the design optimization of the machine tool column and turbine stator blades. The proposed method exhibits higher sampling efficiency, particularly in scenarios with limited samples.