<p>In this paper, we study second-order methods for solving convex-strongly concave minimax problems, which have attracted much attention in recent years due to their wide application in machine learning and related fields. We propose a minimax Levenberg–Marquardt (MLM) algorithm to solve convex-strongly concave minimax optimization problems, where the regularization coefficient is proportional to the root mean square of the gradient norm. The iteration complexity of the MLM algorithm to obtain an <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="40305_2025_645_Article_IEq2.gif" Format="GIF" Height="10" Rendition="HTML" Resolution="72" Type="Linedraw" Width="11" /> </InlineMediaObject> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-optimal solution is bounded by <InlineEquation ID="IEq3"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="40305_2025_645_Article_IEq3.gif" Format="GIF" Height="21" Rendition="HTML" Resolution="72" Type="Linedraw" Width="153" /> </InlineMediaObject> <EquationSource Format="TEX">\(\mathcal {O}(\ell ^{3/2}\rho ^{1/2}\mu ^{-3/2}\varepsilon ^{-1/2})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="script">O</mi> <mo stretchy="false">(</mo> <msup> <mi>ℓ</mi> <mrow> <mn>3</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msup> <msup> <mi>ρ</mi> <mrow> <mn>1</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msup> <msup> <mi>μ</mi> <mrow> <mo>-</mo> <mn>3</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msup> <msup> <mi>ε</mi> <mrow> <mo>-</mo> <mn>1</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msup> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation>, where <InlineEquation ID="IEq4"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="40305_2025_645_Article_IEq4.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="15" /> </InlineMediaObject> <EquationSource Format="TEX">\(\mu \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>μ</mi> </math></EquationSource> </InlineEquation> is the strongly concave coefficient, and <InlineEquation ID="IEq5"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="40305_2025_645_Article_IEq5.gif" Format="GIF" Height="14" Rendition="HTML" Resolution="72" Type="Linedraw" Width="12" /> </InlineMediaObject> <EquationSource Format="TEX">\(\ell \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ℓ</mi> </math></EquationSource> </InlineEquation> and <InlineEquation ID="IEq6"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="40305_2025_645_Article_IEq6.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="13" /> </InlineMediaObject> <EquationSource Format="TEX">\(\rho \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ρ</mi> </math></EquationSource> </InlineEquation> are the Lipschitz constants of the gradient and Jacobian matrix, respectively. We further develop an accelerated variant of the MLM algorithm (AMLM) through a novel “contracting proximal” framework, which improves the iteration complexity to <InlineEquation ID="IEq7"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="40305_2025_645_Article_IEq7.gif" Format="GIF" Height="22" Rendition="HTML" Resolution="72" Type="Linedraw" Width="123" /> </InlineMediaObject> <EquationSource Format="TEX">\(\tilde{\mathcal {O}}(\ell \rho ^{1/3}\mu ^{-1}\varepsilon ^{-1/3})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mover accent="true"> <mi mathvariant="script">O</mi> <mo stretchy="false">~</mo> </mover> <mrow> <mo stretchy="false">(</mo> <mi>ℓ</mi> <msup> <mi>ρ</mi> <mrow> <mn>1</mn> <mo stretchy="false">/</mo> <mn>3</mn> </mrow> </msup> <msup> <mi>μ</mi> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> <msup> <mi>ε</mi> <mrow> <mo>-</mo> <mn>1</mn> <mo stretchy="false">/</mo> <mn>3</mn> </mrow> </msup> <mo stretchy="false">)</mo> </mrow> </mrow> </math></EquationSource> </InlineEquation>. To the best of our knowledge, the iteration complexity of the proposed AMLM algorithm is the best among the current second-order methods for solving convex-strongly concave minimax problems. Numerical results verify the efficiency of the proposed algorithm.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Minimax Levenberg–Marquardt Algorithms for Convex-Strongly Concave Minimax Problems

  • Jun-Lin Wang,
  • Min-Hao Zhang,
  • Zi Xu

摘要

In this paper, we study second-order methods for solving convex-strongly concave minimax problems, which have attracted much attention in recent years due to their wide application in machine learning and related fields. We propose a minimax Levenberg–Marquardt (MLM) algorithm to solve convex-strongly concave minimax optimization problems, where the regularization coefficient is proportional to the root mean square of the gradient norm. The iteration complexity of the MLM algorithm to obtain an \(\varepsilon \) ε -optimal solution is bounded by \(\mathcal {O}(\ell ^{3/2}\rho ^{1/2}\mu ^{-3/2}\varepsilon ^{-1/2})\) O ( 3 / 2 ρ 1 / 2 μ - 3 / 2 ε - 1 / 2 ) , where \(\mu \) μ is the strongly concave coefficient, and \(\ell \) and \(\rho \) ρ are the Lipschitz constants of the gradient and Jacobian matrix, respectively. We further develop an accelerated variant of the MLM algorithm (AMLM) through a novel “contracting proximal” framework, which improves the iteration complexity to \(\tilde{\mathcal {O}}(\ell \rho ^{1/3}\mu ^{-1}\varepsilon ^{-1/3})\) O ~ ( ρ 1 / 3 μ - 1 ε - 1 / 3 ) . To the best of our knowledge, the iteration complexity of the proposed AMLM algorithm is the best among the current second-order methods for solving convex-strongly concave minimax problems. Numerical results verify the efficiency of the proposed algorithm.