Partitioned Dual Weighting Strategy for Combining Regression Estimates
摘要
Ensemble learning is a powerful paradigm that enhances predictive performance by aggregating multiple base learners. However, conventional ensemble weighting strategies typically focus on either reducing inter-model correlation or emphasizing individual model accuracy, which often results in suboptimal performance in complex, high-dimensional, or noisy environments. In this paper, we propose a novel ensemble combination framework, termed the Partitioned Dual Weighting (PDW), which jointly considers both the correlation structure and local predictive accuracy of base learners. PDW integrates two complementary components: (i) a correlation-based module that constructs more robust eigen-regressors using a shrinkage Tyler’s covariance estimator; and (ii) an accuracy-based module that partitions the feature space and assigns region-specific weights based on local RMSE. The final ensemble weights are adaptively fused, leveraging both global structure and local performance. Experimental results show that PDW achieves superior robustness, lower mean squared error, and improved generalization across heterogeneous prediction scenarios. These findings highlight the effectiveness of jointly modeling correlation and accuracy in ensemble regression tasks.