In this paper, we consider online optimization problems with nonconvex composite objective functions. These problems are particularly relevant in applications such as subspace tracking and sparse recovery, which are crucial in the field of machine learning. By leveraging techniques such as online Newton’s update and scaled proximal mapping, we propose an inexact online proximal Newton’s method. Furthermore, we establish a dynamic regret bound that scales linearly with the cumulative variation between time optima. This bound retains the favorable theoretical properties of online Newton’s method for smooth optimization. Finally, we conduct numerical experiments to empirically validate and demonstrate the effectiveness of the proposed method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dynamic Regret for Online Proximal Newton’s Method

  • Chang He,
  • Zhaoye Pan,
  • Bo Jiang

摘要

In this paper, we consider online optimization problems with nonconvex composite objective functions. These problems are particularly relevant in applications such as subspace tracking and sparse recovery, which are crucial in the field of machine learning. By leveraging techniques such as online Newton’s update and scaled proximal mapping, we propose an inexact online proximal Newton’s method. Furthermore, we establish a dynamic regret bound that scales linearly with the cumulative variation between time optima. This bound retains the favorable theoretical properties of online Newton’s method for smooth optimization. Finally, we conduct numerical experiments to empirically validate and demonstrate the effectiveness of the proposed method.