Scholarlink

<p>This study introduces a novel shrinkage regularization approach for semi-parametric regression trees, combining the strengths of parametric and nonparametric methods to enhance predictive performance and adaptability. The proposed method integrates weighted regression with regularization penalties, leveraging prior estimations and data-driven weighting schemes to mitigate multicollinearity and overfitting. We evaluate the model alongside classical techniques (Ridge, LASSO, Elastic Net) in both parametric and semi-parametric frameworks, incorporating regression trees for nonparametric components. Simulation studies and real-world datasets (Liver Disorders and Computer Hardware) demonstrate the superior performance of the proposed method, particularly in handling multicollinearity and capturing nonlinear relationships. Results show significant improvements in predictive accuracy, as measured by <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(R^2\)</EquationSource> <EquationSource Format="MATHML"><math> <msup> <mi>R</mi> <mn>2</mn> </msup> </math></EquationSource> </InlineEquation>, MSE, AIC, and BIC, with tree-augmented variants (e.g., NR-Trees) achieving the highest performance. This work bridges the gap between interpretability and flexibility, offering a robust tool for complex data structures in fields like econometrics, epidemiology, and machine learning.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A note on shrinkage regularization in semi-parametric regression trees

  • 期刊论文

  • 发表日期：

    2026年7月3日

查看全文
  • Hamid Karamikabir,
  • Mohammad Reza Khalvati Fahliyani

摘要

This study introduces a novel shrinkage regularization approach for semi-parametric regression trees, combining the strengths of parametric and nonparametric methods to enhance predictive performance and adaptability. The proposed method integrates weighted regression with regularization penalties, leveraging prior estimations and data-driven weighting schemes to mitigate multicollinearity and overfitting. We evaluate the model alongside classical techniques (Ridge, LASSO, Elastic Net) in both parametric and semi-parametric frameworks, incorporating regression trees for nonparametric components. Simulation studies and real-world datasets (Liver Disorders and Computer Hardware) demonstrate the superior performance of the proposed method, particularly in handling multicollinearity and capturing nonlinear relationships. Results show significant improvements in predictive accuracy, as measured by \(R^2\) R 2 , MSE, AIC, and BIC, with tree-augmented variants (e.g., NR-Trees) achieving the highest performance. This work bridges the gap between interpretability and flexibility, offering a robust tool for complex data structures in fields like econometrics, epidemiology, and machine learning.