Ensemble learning is a powerful paradigm that enhances predictive performance by aggregating multiple base learners. However, conventional ensemble weighting strategies typically focus on either reducing inter-model correlation or emphasizing individual model accuracy, which often results in suboptimal performance in complex, high-dimensional, or noisy environments. In this paper, we propose a novel ensemble combination framework, termed the Partitioned Dual Weighting (PDW), which jointly considers both the correlation structure and local predictive accuracy of base learners. PDW integrates two complementary components: (i) a correlation-based module that constructs more robust eigen-regressors using a shrinkage Tyler’s covariance estimator; and (ii) an accuracy-based module that partitions the feature space and assigns region-specific weights based on local RMSE. The final ensemble weights are adaptively fused, leveraging both global structure and local performance. Experimental results show that PDW achieves superior robustness, lower mean squared error, and improved generalization across heterogeneous prediction scenarios. These findings highlight the effectiveness of jointly modeling correlation and accuracy in ensemble regression tasks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Partitioned Dual Weighting Strategy for Combining Regression Estimates

  • Liusha Yang,
  • Siqi Zhao,
  • Lingjie Li,
  • Shibo Chen

摘要

Ensemble learning is a powerful paradigm that enhances predictive performance by aggregating multiple base learners. However, conventional ensemble weighting strategies typically focus on either reducing inter-model correlation or emphasizing individual model accuracy, which often results in suboptimal performance in complex, high-dimensional, or noisy environments. In this paper, we propose a novel ensemble combination framework, termed the Partitioned Dual Weighting (PDW), which jointly considers both the correlation structure and local predictive accuracy of base learners. PDW integrates two complementary components: (i) a correlation-based module that constructs more robust eigen-regressors using a shrinkage Tyler’s covariance estimator; and (ii) an accuracy-based module that partitions the feature space and assigns region-specific weights based on local RMSE. The final ensemble weights are adaptively fused, leveraging both global structure and local performance. Experimental results show that PDW achieves superior robustness, lower mean squared error, and improved generalization across heterogeneous prediction scenarios. These findings highlight the effectiveness of jointly modeling correlation and accuracy in ensemble regression tasks.