To Smoothly Go Where No Model Has Gone Before: Pareto Tournaments, Model Curvature, and Alternating Objectives

A good model is simple, accurate and predicts well against new regions of parameter space. Multi-objectiveMulti-objectiveselection during search can focus the evolutionary effort on models which satisfy these objectives; however, the curse-of-dimensionalitydimensionalityresults reduced selectivity focus when we have too many objectives. Herein, we discuss the motivation for a new measure for model smoothnesssmoothnessand explore its incorporation in symbolic regressionsymbolic regressionmodel development. The introduction of this new smoothnesssmoothnessobjective allowed us to revisit previous implementation choices so we discuss our learning here related to alternating objectivesalternating objectives, new Pareto tournamentPareto tournamentPareto tournamentvariants, characterizing model complexity (functional and structural), ensembles, model-age as a selection criterion, and the use of synthetic data in search. We ensure computational efficiency and maintenance of model diversity throughout since these are essential for commercial applications.