Scholarlink

<p>The conjugate gradient (CG) algorithm is widely recognized for its efficiency and simplicity in solving nonlinear equations as well as smooth and nonsmooth optimization problems. Nevertheless, its application to stochastic optimization remains relatively underexplored. In this paper, we propose a novel stochastic CG algorithm that integrates a variance reduction mechanism with an inexact line search strategy to effectively address stochastic optimization problems. Under standard assumptions, we establish that the proposed algorithm converges almost surely to a stationary point. Moreover, we provide a detailed complexity analysis and show that the algorithm attains a linear convergence rate in the strongly convex setting. The incorporation of variance reduction significantly mitigates the adverse effects of stochastic gradient noise, including slow convergence and variance amplification. Meanwhile, the inexact line search adaptively adjusts the step size, thereby enhancing computational efficiency without compromising theoretical guarantees. Extensive numerical experiments on both nonconvex and strongly convex problems validate the effectiveness of the proposed framework, demonstrating its superior performance compared with existing stochastic optimization algorithms.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A stochastic conjugate gradient algorithm with Armijo technique for machine learning

  • 期刊论文

  • 发表日期：

    2026年3月23日

查看全文
  • Gonglin Yuan,
  • Yingjie Zhou,
  • Mengxiang Zhang,
  • Junyu Lu

摘要

The conjugate gradient (CG) algorithm is widely recognized for its efficiency and simplicity in solving nonlinear equations as well as smooth and nonsmooth optimization problems. Nevertheless, its application to stochastic optimization remains relatively underexplored. In this paper, we propose a novel stochastic CG algorithm that integrates a variance reduction mechanism with an inexact line search strategy to effectively address stochastic optimization problems. Under standard assumptions, we establish that the proposed algorithm converges almost surely to a stationary point. Moreover, we provide a detailed complexity analysis and show that the algorithm attains a linear convergence rate in the strongly convex setting. The incorporation of variance reduction significantly mitigates the adverse effects of stochastic gradient noise, including slow convergence and variance amplification. Meanwhile, the inexact line search adaptively adjusts the step size, thereby enhancing computational efficiency without compromising theoretical guarantees. Extensive numerical experiments on both nonconvex and strongly convex problems validate the effectiveness of the proposed framework, demonstrating its superior performance compared with existing stochastic optimization algorithms.