Regression, as a statistical learning process seeks to minimize estimation error by modelling the relationship between independent variables, but Overfitting can result in minimal training loss with significant testing error. Although existing literature offers solutions like Ridge ( \(\ell _2\) Norm) and Lasso ( \(\ell _1\) Norm) Regression, these methods lack adaptability and precision. In this research, a modified \(\ell _{2}\) Norm-based regularization is proposed, with an ellipsoid restrictive region for the choice of parameters with the least residual error. The modified \(\ell _{2}\) regularization has been proved to possess a more adaptable and precise geometry, a better balance in the bias-variance trade-off, and further leads to tighter generalization bounds, as per the Vapnik-Chervonenkis dimension. The efficacy of this modified \(\ell _{2}\) Norm based regularization in addressing overfitting is compared with various norm-based regularizations including the \(\ell _1\) , \(\ell _2\) , and a few state-of-the-art paradigms following its applicability on a few real-time datasets with cross-domain applicability.
