Stochastic Functions Learning from Distribution-Driven Data: Generalization Bound and Algorithms

This paper delves into the learning of stochastic functions from distribution-driven data. Utilizing the empirical risk minimization (ERM) approach, we construct a stochastic minimax optimization problem from the distribution-driven data to recover the target random function. The generalization bound of the ERM approach is established based on annealed VC entropy. When the hypothesis set of functions is parameterized by a vector over a compact set, the optimization problem is simplified to a finite-dimensional stochastic composite minimax problem. This problem can be solved through a stochastic compositional gradient descent ascent method. Under mild conditions, the method is proved to converge almost surely to an optimal solution and achieve a sublinear convergence rate.