<p>In a Hilbert framework, for convex differentiable optimization, we consider accelerated gradient methods obtained by combining temporal scaling and averaging techniques with Tikhonov regularization. We start from the continuous steepest descent dynamic with an additional Tikhonov regularization term whose coefficient vanishes asymptotically. We provide an extensive Lyapunov analysis of this first-order evolution equation. Then we apply to this dynamic the method of time scaling and averaging recently introduced by Attouch, Boţ and Nguyen. We thus obtain an inertial dynamic which involves viscous damping associated with Nesterov’s method, implicit Hessian damping and Tikhonov regularization. Under an appropriate setting of the parameters, just using Jensen’s inequality, without the need for another Lyapunov analysis, we show that the trajectories have at the same time several remarkable properties: they provide a rapid convergence of values, fast convergence of the gradients to zero, and strong convergence to the minimum norm minimizer. Then we extend the results to the nonsmooth setting. These results complete and improve the previous results obtained by the authors.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Accelerated gradient methods with strong convergence to the minimum norm minimizer: a dynamic approach combining time scaling, averaging, and Tikhonov regularization

  • Hedy Attouch,
  • Zaki Chbani,
  • Hassan Riahi

摘要

In a Hilbert framework, for convex differentiable optimization, we consider accelerated gradient methods obtained by combining temporal scaling and averaging techniques with Tikhonov regularization. We start from the continuous steepest descent dynamic with an additional Tikhonov regularization term whose coefficient vanishes asymptotically. We provide an extensive Lyapunov analysis of this first-order evolution equation. Then we apply to this dynamic the method of time scaling and averaging recently introduced by Attouch, Boţ and Nguyen. We thus obtain an inertial dynamic which involves viscous damping associated with Nesterov’s method, implicit Hessian damping and Tikhonov regularization. Under an appropriate setting of the parameters, just using Jensen’s inequality, without the need for another Lyapunov analysis, we show that the trajectories have at the same time several remarkable properties: they provide a rapid convergence of values, fast convergence of the gradients to zero, and strong convergence to the minimum norm minimizer. Then we extend the results to the nonsmooth setting. These results complete and improve the previous results obtained by the authors.