Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

This paper presents a novel two-stage progressive search approach with unsupervised feature learning and Q-learning (TSLL) to enhance surrogate-assisted evolutionary optimization for medium-scale expensive problems. The method attempts to address the challenges posed by multi-polar and multi-variable coupling properties in such problems. During the iteration, TSLL splits the optimization process into two distinct stages. First, two unsupervised feature learning techniques: principal component analysis (PCA) and Autoencoder, are utilized to improve the representation of potential optimal samples in the solution space. PCA is used for feature reduction, extracting the most important features. On the other hand, Autoencoder focuses on reconstructing features within the medium-scale solution space. To ensure comprehensive exploration of the entire solution space, TSLL employs two distinct surrogate modeling approaches along with Q-learning in the second stage to dynamically select the mutation strategy for the differential evolution operator. Numerous experiments demonstrate the superiority of TSLL over five state-of-the-art surrogate-assisted approaches and two sophisticated evolutionary algorithms including the winner of CEC 2017 on medium-scale benchmark problems and a wind farm layout problem.