A Study on Accelerating of Inertial Newton Algorithm for Neural Network Training

This paper describes the novel study on accelerating INertial Newton Algorithm (INNA) for neural network training. Recently, INNA, a dynamic system of optimization methods, has been proposed and applied to neural network training. INNA combines the ideas of Newton and the Inertial methods into a dynamical system and expresses them as differential equations. This paper proposes a new training algorithm called Nesterov’s Accelerated Dynamical InertiAl Newton method (NADIAN), which accelerates INNA by introducing Nesterov’s accelerated gradient. Finally, the proposed method is applied to neural network training and verified.