Sharpness-Aware Minimization in Genetic Programming
摘要
Sharpness-Aware Minimization (SAM) wasSharpness recently introduced as a regularization procedure for training deep neural networks. It simultaneously minimizes the fitness (or loss) function and fitness sharpnessSharpness. The latter serves as a measure of the nonlinear behavior of a solution guiding toward solutions that lie in neighborhoods with uniformly similar loss values across all fitness cases. In this contribution we adapt SAM for tree Genetic Programming (TGP) by exploring the semanticSemantics neighborhoods of solutions using two simple approaches. By perturbing input and output of program trees, sharpnessSharpness can be estimated and used as a second optimization criterion during the evolution. To better understand the impact of this variant of SAM on TGP, we collect numerous indicators of the evolutionary process, including generalization ability, complexity, diversity, and a recently proposed genotype-phenotype mappingGenotype-phenotype mapping to study the amount of redundancy in trees. The experimental results demonstrate that using any of the two proposed SAM adaptations in TGP allows (i) a significant reduction of tree sizes in the population and (ii) a decrease in redundancy of the trees. When assessed on real-world benchmarks, the generalization ability of solutions does not deteriorate.