Scholarlink

<p>L-BFGS is one of the widely used quasi-Newton methods. Instead of explicitly storing an approximation <i>H</i> of the inverse Hessian, L-BFGS keeps a limited number of vectors that can be used for computing the product of <i>H</i> by the gradient. However, this computation is sequential, each step depending on the outcome of the previous step. To solve this problem, we propose the Direct L-BFGS (DirL-BFGS) method that, seeing <i>H</i> as a linear operator, directly stores a low-rank plus diagonal (LRPD) representation of <i>H</i>. Employing the LRPD representation enables us to leverage the benefits of vector processing, leading to accelerating and parallelizing the calculations in the form of single instruction, multiple data. We evaluate our proposed method on different quadratic optimization problems and several regression and classification tasks with neural networks. Numerical results show that DirL-BFGS is faster overall than L-BFGS.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Speeding up L-BFGS by direct approximation of the inverse Hessian matrix

  • 期刊论文

  • 发表日期：

    2025年2月24日

查看全文
  • Ashkan Sadeghi-Lotfabadi,
  • Kamaledin Ghiasi-Shirazi

摘要

L-BFGS is one of the widely used quasi-Newton methods. Instead of explicitly storing an approximation H of the inverse Hessian, L-BFGS keeps a limited number of vectors that can be used for computing the product of H by the gradient. However, this computation is sequential, each step depending on the outcome of the previous step. To solve this problem, we propose the Direct L-BFGS (DirL-BFGS) method that, seeing H as a linear operator, directly stores a low-rank plus diagonal (LRPD) representation of H. Employing the LRPD representation enables us to leverage the benefits of vector processing, leading to accelerating and parallelizing the calculations in the form of single instruction, multiple data. We evaluate our proposed method on different quadratic optimization problems and several regression and classification tasks with neural networks. Numerical results show that DirL-BFGS is faster overall than L-BFGS.