<p>Multi-objective genetic programming-based feature construction has emerged as a powerful approach to enhancing regression performance. While this approach has demonstrated considerable success across various domains, overfitting remains a significant challenge, particularly when dealing with limited and/or noisy training data. To address this, researchers have developed multi-objective methods that balance model complexity and accuracy. However, these methods invariably produce a Pareto front containing multiple solutions with different trade-offs, raising the critical question of how to select the most appropriate model for deployment. In this paper, we propose a novel Adaptive Complexity Knee Point (ACKP) selection strategy for evolutionary multi-objective feature construction in regression tasks. Our approach adaptively selects between minimal complexity knee points, which favor the simplest knee solution, and traditional knee points, based on estimated dataset difficulty. This adaptive mechanism reduces overfitting in noisy scenarios while mitigating underfitting in low-noise datasets by allowing more complex models when appropriate. Comprehensive experiments on 58 real-world datasets demonstrate that ACKP significantly outperforms nine established model selection strategies, and that genetic programming with ACKP outperforms several mainstream machine learning algorithms, particularly when dealing with sample-limited and noisy datasets, highlighting its practical value in real-world problems.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive complexity knee point selection in multi-objective genetic programming for improving generalization of evolutionary feature construction in regression

  • Hengzhe Zhang,
  • Qi Chen,
  • Bing Xue,
  • Wolfgang Banzhaf,
  • Mengjie Zhang

摘要

Multi-objective genetic programming-based feature construction has emerged as a powerful approach to enhancing regression performance. While this approach has demonstrated considerable success across various domains, overfitting remains a significant challenge, particularly when dealing with limited and/or noisy training data. To address this, researchers have developed multi-objective methods that balance model complexity and accuracy. However, these methods invariably produce a Pareto front containing multiple solutions with different trade-offs, raising the critical question of how to select the most appropriate model for deployment. In this paper, we propose a novel Adaptive Complexity Knee Point (ACKP) selection strategy for evolutionary multi-objective feature construction in regression tasks. Our approach adaptively selects between minimal complexity knee points, which favor the simplest knee solution, and traditional knee points, based on estimated dataset difficulty. This adaptive mechanism reduces overfitting in noisy scenarios while mitigating underfitting in low-noise datasets by allowing more complex models when appropriate. Comprehensive experiments on 58 real-world datasets demonstrate that ACKP significantly outperforms nine established model selection strategies, and that genetic programming with ACKP outperforms several mainstream machine learning algorithms, particularly when dealing with sample-limited and noisy datasets, highlighting its practical value in real-world problems.