Dynamic Regret for Online Proximal Newton’s Method
摘要
In this paper, we consider online optimization problems with nonconvex composite objective functions. These problems are particularly relevant in applications such as subspace tracking and sparse recovery, which are crucial in the field of machine learning. By leveraging techniques such as online Newton’s update and scaled proximal mapping, we propose an inexact online proximal Newton’s method. Furthermore, we establish a dynamic regret bound that scales linearly with the cumulative variation between time optima. This bound retains the favorable theoretical properties of online Newton’s method for smooth optimization. Finally, we conduct numerical experiments to empirically validate and demonstrate the effectiveness of the proposed method.