Unified Analysis of SGD-Type Methods
摘要
This chapter surveys a unified framework for the analysis of SGD-type methods for smooth and strongly convex optimization, with the main results based on the unified theory introduced in [21]. Under a simple parametric assumption on stochastic gradient estimators, a wide range of algorithms—including classical SGD, methods with expected smoothness, interpolation regimes, variance reduction, coordinate descent, and distributed methods with compression—can be analyzed within a single convergence proof. The framework yields linear convergence guarantees in expectation and recovers known optimal rates in many standard settings. Extensions, limitations, and alternative analytical approaches are briefly discussed, highlighting the framework’s pedagogical value and its role in systematizing modern stochastic optimization methods.