A regression-based L2-norm twin support vector machine for binary classification
摘要
The twin support vector machine (TWSVM) has emerged as a popular advancement of the support vector machine (SVM) due to its efficiency and broad applicability. TWSVM is a binary classifier that identifies two non-parallel hyperplanes, each closest to its class and farthest from the opposing class. This paper proposes a novel twin support vector machine called RL2-TWSVM to address the issues in TWSVM: (1) The complexity of calculating the inversion of a large matrix and, consequently, the problem of ill-conditioned matrix inversion; (2) Sensitivity to noises around the decision boundary due to using the hinge loss function; (3) difficulty adapting the linear model to the nonlinear one using the kernel trick employed in SVM. RL2-TWSVM identifies the optimal hyperplane for each class by adjusting a flexible boundary with a minimum distance around the corresponding data while maximizing the margin from the data of the opposing class. The new optimization problem introduces two key innovations: (1) replacement of hinge loss with a quadratic ε-insensitive loss function to improve robustness against boundary noise; (2) incorporation of L2-norm regularizations to reduce computational complexity and enhance stability. The new primal objective function determines the optimal parameter ε automatically and is solved in dual form using the successive over relaxation method. The experimental results on several artificial and real-world data sets indicate that, in addition to overcoming the issues of the traditional TWSVM, RL2-TWSVM has also demonstrated superior performance over TWSVM and other state-of-the-art methods, such as LPTWSVM and Spin-FITBSVM.