Stochastic DC algorithms for general stochastic DC programs with machine learning applications
摘要
We consider the class of Difference-of-Convex-functions (DC) constrained stochastic DC programs, which are optimization problems where the objective function is the expected value of a stochastic DC function based on a probability distribution, while the constraint functions are (deterministic) DC. While a variety of methods have been proposed to solve convex constrained stochastic optimization problems, there are few approaches specifically designed for solving nonconvex and nonsmooth constrained stochastic programs. Recognized in deterministic optimization literature as one of the few efficient algorithms for large-scale nonconvex and nonsmooth problems, the DC algorithm (DCA) will be leveraged in this work. Using penalty techniques, we transform nonconvex constrained stochastic DC programs into convex constrained stochastic DC programs and introduce novel stochastic DC algorithms to solve the resulting problems. The convergence analysis of the proposed algorithms is thoroughly examined. These algorithms are applied to two significant machine learning problems: Expected Sparse Principal Component Analysis and Binary Classification. Numerical experiments are conducted to evaluate their performance.