Support vector machine with Universum data (USVM) is an effective algorithm which utilizes the prior information embedded in Universum data to improve its generalization ability. However, the \(l_{1}\) -loss adopted by USVM has an irrationality in geometric definition, which makes it unable to accurately reflect the location information of violated samples, thus degrading the model’s performance to some extent. To address the above limitation, we propose a novel elastic net support vector machine with Universum data (ENUSVM). It adopts the elastic net loss instead of \(l_{1}\) -loss to achieve a one-to-one mapping between the training samples and the slack variables. This improvement makes it more reasonable in geometric definition and thus improves the performance. Further, we derive the violation tolerance upper bound for labeled samples and Universum samples to better interpret the numerical relationship between the distance of violated samples and the slack variables. Experiments on three artificial datasets and twenty benchmark datasets prove that our proposed ENUSVM performs better. Finally, to enhance the robustness of USVM, we incorporate the RoBoSS loss and propose the least squares RoBoSS USVM ( \( \mathcal {L}_{RoBoSS}\) -USVM). The performance is evaluated on five diverse UCI and KEEL datasets (with and without noise).
