Soft adaptive node-level stabilization (SANLS): enhancing decision tree stability with information gain optimization and advanced splitting criteria
摘要
Decision trees are widely used in machine learning for their simplicity and interpretability. However, traditional methods often encounter challenges such as instability, overfitting, and inefficiency, particularly in noisy or complex datasets. This study introduces the Soft Adaptive Node-Level Stabilization (SANLS) framework, which replaces rigid decision boundaries with smooth, probabilistic transitions using soft decision functions. By integrating advanced splitting criteria such as gain ratio, entropy, Chi-square, and Gini index and optimizing information gain, SANLS enhances decision tree stability and accuracy. A dynamic bootstrapping mechanism further improves computational efficiency by determining the optimal number of iterations needed for reliable splits. Experimental results on UCI respiratory multiple benchmark datasets such as Sonar, Spam base, Ionosphere, Letter, Shuttle, Car, and Yeast datasets demonstrate that SANLS consistently outperforms traditional methods, achieving higher accuracy, stability, and computational efficiency. Moreover, SANLS leverages its differentiable structure to enable seamless integration into neural network architectures, addressing limitations in generative adversarial models through the introduction of soft decision forests. This study establishes SANLS as a robust, scalable solution for decision tree optimization, paving the way for improved performance in diverse domains.