An NEPv Approach for Orthogonal CCA with Matrix (2, 1)-norm Regularization with Application to Feature Selection
摘要
The matrix (2, 1)-norm regularization is an effective tool to generate row-sparse projections, which play critical roles in terms of feature selection in data science applications. This paper is concerned with orthogonal canonical correlation analysis (OCCA) with the matrix (2, 1)-norm regularization, arising from a novel feature selection model to be proposed. The model is a nonsmooth maximization problem on the Stiefel manifold, and a practical NEPv approach (nonlinear eigenvalue problem with eigenvector dependency) is proposed to efficiently solve the problem. It is proved that the NEPv approach always produces a sequence of approximations with monotonically increasing objective values and is globally convergent to a stationary point. As an application to feature selection, the proposed approach is demonstrated on several real-world datasets in the public domain. Numerical results show that the efficiency of the proposed numerical approach for high dimensional data and the superior performance of the feature selection model, compared to existing feature selection methods.