Scholarlink

<p>This paper studies a class of nonconvex stochastic optimization problems involving simple nonsmooth terms, which commonly arise in machine learning and signal processing applications. We propose a unified mini-batch stochastic accelerated (UMSA) algorithm that handles both convex and nonconvex settings within a single framework. The algorithm achieves the known optimal stochastic first-order oracle (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\mathcal {SFO}\)</EquationSource> <EquationSource Format="MATHML"><math> <mi mathvariant="script">SFO</mi> </math></EquationSource> </InlineEquation>) complexity order <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\mathcal{{O}}(\frac{1}{\epsilon ^2})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="script">O</mi> <mo stretchy="false">(</mo> <mfrac> <mn>1</mn> <msup> <mi>ϵ</mi> <mn>2</mn> </msup> </mfrac> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> in both cases, and exhibits greater flexibility in parameter selection compared to existing methods. Notably, the proposed UMSA algorithm subsumes several existing algorithms as special cases, offering a unifying perspective on nonsmooth optimization. We provide detailed convergence analysis for both nonconvex and convex scenarios, and show that UMSA is theoretically efficient and practically versatile. Numerical experiments on a nonconvex sparse support vector machine problem validate both the effectiveness and computational efficiency of the proposed UMSA algorithm.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A unified mini-batch stochastic accelerated method for nonconvex stochastic programming

  • 期刊论文

  • 发表日期：

    2025年12月1日

查看全文
  • Ruyu Wang,
  • Cong Liu,
  • Quanwei Gao

摘要

This paper studies a class of nonconvex stochastic optimization problems involving simple nonsmooth terms, which commonly arise in machine learning and signal processing applications. We propose a unified mini-batch stochastic accelerated (UMSA) algorithm that handles both convex and nonconvex settings within a single framework. The algorithm achieves the known optimal stochastic first-order oracle ( \(\mathcal {SFO}\) SFO ) complexity order \(\mathcal{{O}}(\frac{1}{\epsilon ^2})\) O ( 1 ϵ 2 ) in both cases, and exhibits greater flexibility in parameter selection compared to existing methods. Notably, the proposed UMSA algorithm subsumes several existing algorithms as special cases, offering a unifying perspective on nonsmooth optimization. We provide detailed convergence analysis for both nonconvex and convex scenarios, and show that UMSA is theoretically efficient and practically versatile. Numerical experiments on a nonconvex sparse support vector machine problem validate both the effectiveness and computational efficiency of the proposed UMSA algorithm.