We propose an adaptive zeroth-order method for minimizing differentiable functions with L-Lipschitz continuous gradients. The method is designed to take advantage of the eventual compressibility of the gradient of the objective function, but it does not require knowledge of the approximate sparsity level s or the Lipschitz constant L of the gradient. We show that the new method performs no more than <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$\mathcal {O}\left( n^{2}\epsilon ^{-2}\right) $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="script">O</mi> <mfenced close=")" open="("> <msup> <mi>n</mi> <mn>2</mn> </msup> <msup> <mi>ϵ</mi> <mrow> <mo>-</mo> <mn>2</mn> </mrow> </msup> </mfenced> </mrow> </math></EquationSource> </InlineEquation> function evaluations to find an <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$\epsilon $</EquationSource> <EquationSource Format="MATHML"><math> <mi>ϵ</mi> </math></EquationSource> </InlineEquation>-approximate stationary point of an objective function with n variables. Assuming additionally that the gradients of the objective function are compressible, we obtain an improved complexity bound of <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\mathcal {O}\left( s\log \left( n\right) \epsilon ^{-2}\right) $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="script">O</mi> <mfenced close=")" open="("> <mi>s</mi> <mo>log</mo> <mfenced close=")" open="("> <mi>n</mi> </mfenced> <msup> <mi>ϵ</mi> <mrow> <mo>-</mo> <mn>2</mn> </mrow> </msup> </mfenced> </mrow> </math></EquationSource> </InlineEquation> function evaluations, which holds with high probability. Preliminary numerical results illustrate the efficiency of the proposed method and demonstrate that it can significantly outperform its non-adaptive counterpart.

Fully Adaptive Zeroth-Order Method for Minimizing Functions with Compressible Gradients

期刊论文
发表日期：

2025年10月15日

查看全文

Geovani N. Grapiglia,
Daniel McKenzie

摘要

We propose an adaptive zeroth-order method for minimizing differentiable functions with L-Lipschitz continuous gradients. The method is designed to take advantage of the eventual compressibility of the gradient of the objective function, but it does not require knowledge of the approximate sparsity level s or the Lipschitz constant L of the gradient. We show that the new method performs no more than $\mathcal {O}\left( n^{2}\epsilon ^{-2}\right) $ $O (n^{2}, ϵ^{- 2})$ function evaluations to find an $\epsilon $ $ϵ$ -approximate stationary point of an objective function with n variables. Assuming additionally that the gradients of the objective function are compressible, we obtain an improved complexity bound of $\mathcal {O}\left( s\log \left( n\right) \epsilon ^{-2}\right) $ $O (s log (n) ϵ^{- 2})$ function evaluations, which holds with high probability. Preliminary numerical results illustrate the efficiency of the proposed method and demonstrate that it can significantly outperform its non-adaptive counterpart.