The application of Goeken–Johnson’s Runge–Kutta methods in unconstrained convex optimization

The paper is devoted to the construction and analysis of gradient methods for convex optimization based on the explicit Goeken–Johnson’s Runge–Kutta methods for solving the Cauchy problem. Such two-step methods are based on the finite-difference approximation of solution derivatives and need fewer stages than standard Runge–Kutta methods. For quadratic problems, theorems on the convergence conditions and optimal stepsize for third- and fourth-order methods are presented. For the class of smooth convex functions, restricted by some assumptions, accelerated methods based on application to the Cauchy problem for non-autonomous system of second-order ordinary differential equations are constructed. The theorem on the convergence rate of accelerated methods is presented. As it is demonstrated for higher-order methods, the convergence rate can be better than for Nesterov’s accelerated gradient method, applied to functions from the considered class. Theoretical results are supported by numerical experiments for problems, which arise in different applications. Following problems are considered: minimization of convex quadratic and non-quadratic functions; minimization of integral functional; logistic regression problem; training of feedforward neural network. As it is demonstrated, proposed methods converge faster than the gradient descent method and Nesterov’s method. According to fewer number of stages, constructed methods require less computational time in comparison with algorithms based on standard explicit Runge–Kutta schemes.