Scholarlink

<p>We develop and analyze stochastic variants of ISTA and a full backtracking FISTA algorithms (Beck and Teboulle in SIAM J Imag Sci 2(1):183–202, 2009; Scheinberg et al. in Found Comput Math 14(3):389–417, 2014) for composite optimization without the assumption that stochastic gradient is an unbiased estimator. This work extends analysis of inexact fixed step ISTA/FISTA in Schmidt et al. (Convergence rates of inexact proximal-gradient methods for convex optimization, 2022. <a href="http://arxiv.org/abs/1109.2415">arXiv:1109.2415</a>) to the case of stochastic gradient estimates and adaptive step-size parameter chosen by backtracking. It also extends the framework for analyzing stochastic line-search method in Cartis and Scheinberg (Math Program 169(2):337-375, 2018) to the proximal gradient framework as well as to the accelerated first order methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimization

  • 期刊论文

  • 发表日期：

    2025年2月14日

查看全文
  • Lam M. Nguyen,
  • Katya Scheinberg,
  • Trang H. Tran

摘要

We develop and analyze stochastic variants of ISTA and a full backtracking FISTA algorithms (Beck and Teboulle in SIAM J Imag Sci 2(1):183–202, 2009; Scheinberg et al. in Found Comput Math 14(3):389–417, 2014) for composite optimization without the assumption that stochastic gradient is an unbiased estimator. This work extends analysis of inexact fixed step ISTA/FISTA in Schmidt et al. (Convergence rates of inexact proximal-gradient methods for convex optimization, 2022. arXiv:1109.2415) to the case of stochastic gradient estimates and adaptive step-size parameter chosen by backtracking. It also extends the framework for analyzing stochastic line-search method in Cartis and Scheinberg (Math Program 169(2):337-375, 2018) to the proximal gradient framework as well as to the accelerated first order methods.