Scholarlink

<p>This paper proposes a homogeneous second-order descent framework (HSODF) for nonconvex and convex optimization based on the generalized homogeneous model (GHM). In comparison to the Newton steps, the GHM can be solved by extremal symmetric eigenvalue procedures and thus grant an advantage in ill-conditioned problems. Moreover, GHM extends the ordinary homogeneous model (Zhang et al. A homogenous second-order descent method for nonconvex optimization, 2022. <a href="http://arxiv.org/abs/2211.08212">arXiv:2211.08212</a> [math]) to allow adaptiveness in the construction of the aggregated matrix. Consequently, HSODF is able to recover some well-known second-order methods, such as trust-region methods and gradient regularized methods, while maintaining comparable iteration complexity bounds. We also study two specific realizations of HSODF. One is adaptive HSODM, which has a parameter-free <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(O(\epsilon ^{-3/2})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>O</mi> <mo stretchy="false">(</mo> <msup> <mi>ϵ</mi> <mrow> <mo>-</mo> <mn>3</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msup> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> global complexity bound for nonconvex second-order Lipschitz continuous objective functions. The other is homotopy HSODM, which is proven to have a global linear rate of convergence without strong convexity. The efficiency of our approach to high-dimensional and ill-conditioned problems is justified by some preliminary numerical results.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Homogeneous second-order descent framework: a fast alternative to Newton-type methods

  • 期刊论文

  • 发表日期：

    2025年5月15日

查看全文
  • Chang He,
  • Yuntian Jiang,
  • Chuwen Zhang,
  • Dongdong Ge,
  • Bo Jiang,
  • Yinyu Ye

摘要

This paper proposes a homogeneous second-order descent framework (HSODF) for nonconvex and convex optimization based on the generalized homogeneous model (GHM). In comparison to the Newton steps, the GHM can be solved by extremal symmetric eigenvalue procedures and thus grant an advantage in ill-conditioned problems. Moreover, GHM extends the ordinary homogeneous model (Zhang et al. A homogenous second-order descent method for nonconvex optimization, 2022. arXiv:2211.08212 [math]) to allow adaptiveness in the construction of the aggregated matrix. Consequently, HSODF is able to recover some well-known second-order methods, such as trust-region methods and gradient regularized methods, while maintaining comparable iteration complexity bounds. We also study two specific realizations of HSODF. One is adaptive HSODM, which has a parameter-free \(O(\epsilon ^{-3/2})\) O ( ϵ - 3 / 2 ) global complexity bound for nonconvex second-order Lipschitz continuous objective functions. The other is homotopy HSODM, which is proven to have a global linear rate of convergence without strong convexity. The efficiency of our approach to high-dimensional and ill-conditioned problems is justified by some preliminary numerical results.