Newton-Type Methods with the Proximal Gradient Step for Sparse Estimation

In this paper, we propose new methods to efficiently solve convex optimization problems encountered in sparse estimation. These methods include a new quasi-Newton method that avoids computing the Hessian matrix and improves efficiency, and we prove its fast convergence. We also prove the local convergence of the Newton method under the assumption of strong convexity. Our proposed methods offer a more efficient and effective approach, particularly for \(L_1\) $L_1$ regularization and group regularization problems, as they incorporate variable selection with each update. Through numerical experiments, we demonstrate the efficiency of our methods in solving problems encountered in sparse estimation. Our contributions include theoretical guarantees and practical applications for various kinds of problems.