<p>Augmented Lagrangian (AL) methods have proven remarkably useful in solving optimization problems with complicated constraints. The last decade has seen the development of overall complexity guarantees for inexact AL variants. Yet, a crucial gap persists in addressing nonsmooth convex constraints. To this end, we present a smoothed augmented Lagrangian (AL) framework where nonsmooth terms are progressively smoothed with a smoothing parameter <InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq1.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="19" /> </InlineMediaObject> <EquationSource Format="TEX">\(\eta _k\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>η</mi> <mi>k</mi> </msub> </math></EquationSource> </InlineEquation>. The resulting AL subproblems are <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq1.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="19" /> </InlineMediaObject> <EquationSource Format="TEX">\(\eta _k\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>η</mi> <mi>k</mi> </msub> </math></EquationSource> </InlineEquation>-smooth, allowing for leveraging accelerated schemes. By a careful selection of the inexactness level <InlineEquation ID="IEq3"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq3.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="17" /> </InlineMediaObject> <EquationSource Format="TEX">\(\epsilon _k\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>ϵ</mi> <mi>k</mi> </msub> </math></EquationSource> </InlineEquation> (for inexact subproblem resolution), the penalty parameter <InlineEquation ID="IEq4"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq4.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="20" /> </InlineMediaObject> <EquationSource Format="TEX">\(\rho _k\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>ρ</mi> <mi>k</mi> </msub> </math></EquationSource> </InlineEquation>, and smoothing parameter <InlineEquation ID="IEq5"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq1.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="19" /> </InlineMediaObject> <EquationSource Format="TEX">\(\eta _k\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>η</mi> <mi>k</mi> </msub> </math></EquationSource> </InlineEquation> at epoch <i>k</i>, we derive rate and complexity guarantees of <InlineEquation ID="IEq6"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq6.gif" Format="GIF" Height="22" Rendition="HTML" Resolution="72" Type="Linedraw" Width="72" /> </InlineMediaObject> <EquationSource Format="TEX">\(\tilde{\mathcal {O}}(1/\varvec{\varepsilon }^{3/2})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mover accent="true"> <mi mathvariant="script">O</mi> <mo stretchy="false">~</mo> </mover> <mrow> <mo stretchy="false">(</mo> <mn>1</mn> <mo stretchy="false">/</mo> <msup> <mrow> <mi mathvariant="bold-italic">ε</mi> </mrow> <mrow> <mn>3</mn> <mo stretchy="false">/</mo> <mn>2</mn> </mrow> </msup> <mo stretchy="false">)</mo> </mrow> </mrow> </math></EquationSource> </InlineEquation> and <InlineEquation ID="IEq7"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq7.gif" Format="GIF" Height="22" Rendition="HTML" Resolution="72" Type="Linedraw" Width="54" /> </InlineMediaObject> <EquationSource Format="TEX">\(\tilde{\mathcal {O}}(1/\varvec{\varepsilon })\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mover accent="true"> <mi mathvariant="script">O</mi> <mo stretchy="false">~</mo> </mover> <mrow> <mo stretchy="false">(</mo> <mn>1</mn> <mo stretchy="false">/</mo> <mrow> <mi mathvariant="bold-italic">ε</mi> </mrow> <mo stretchy="false">)</mo> </mrow> </mrow> </math></EquationSource> </InlineEquation> in convex and strongly convex regimes for computing an <InlineEquation ID="IEq8"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq8.gif" Format="GIF" Height="10" Rendition="HTML" Resolution="72" Type="Linedraw" Width="13" /> </InlineMediaObject> <EquationSource Format="TEX">\(\varvec{\varepsilon }\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="bold-italic">ε</mi> </mrow> </math></EquationSource> </InlineEquation>-optimal solution, when <InlineEquation ID="IEq9"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq4.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="20" /> </InlineMediaObject> <EquationSource Format="TEX">\(\rho _k\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>ρ</mi> <mi>k</mi> </msub> </math></EquationSource> </InlineEquation> increases at a geometric rate, a significant improvement over the best available guarantees for AL schemes for convex programs with nonsmooth constraints. Analogous guarantees are developed for settings with <InlineEquation ID="IEq10"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq10.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="51" /> </InlineMediaObject> <EquationSource Format="TEX">\(\rho _k = \rho \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msub> <mi>ρ</mi> <mi>k</mi> </msub> <mo>=</mo> <mi>ρ</mi> </mrow> </math></EquationSource> </InlineEquation> as well as <InlineEquation ID="IEq11"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="10915_2025_2934_Article_IEq11.gif" Format="GIF" Height="12" Rendition="HTML" Resolution="72" Type="Linedraw" Width="50" /> </InlineMediaObject> <EquationSource Format="TEX">\(\eta _k = \eta \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msub> <mi>η</mi> <mi>k</mi> </msub> <mo>=</mo> <mi>η</mi> </mrow> </math></EquationSource> </InlineEquation>. Preliminary numerics on a fused Lasso problem display promise.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Smoothed Augmented Lagrangian Framework for Convex Optimization with Nonsmooth Constraints

  • Peixuan Zhang,
  • Uday V. Shanbhag,
  • Ethan X. Fang

摘要

Augmented Lagrangian (AL) methods have proven remarkably useful in solving optimization problems with complicated constraints. The last decade has seen the development of overall complexity guarantees for inexact AL variants. Yet, a crucial gap persists in addressing nonsmooth convex constraints. To this end, we present a smoothed augmented Lagrangian (AL) framework where nonsmooth terms are progressively smoothed with a smoothing parameter \(\eta _k\) η k . The resulting AL subproblems are \(\eta _k\) η k -smooth, allowing for leveraging accelerated schemes. By a careful selection of the inexactness level \(\epsilon _k\) ϵ k (for inexact subproblem resolution), the penalty parameter \(\rho _k\) ρ k , and smoothing parameter \(\eta _k\) η k at epoch k, we derive rate and complexity guarantees of \(\tilde{\mathcal {O}}(1/\varvec{\varepsilon }^{3/2})\) O ~ ( 1 / ε 3 / 2 ) and \(\tilde{\mathcal {O}}(1/\varvec{\varepsilon })\) O ~ ( 1 / ε ) in convex and strongly convex regimes for computing an \(\varvec{\varepsilon }\) ε -optimal solution, when \(\rho _k\) ρ k increases at a geometric rate, a significant improvement over the best available guarantees for AL schemes for convex programs with nonsmooth constraints. Analogous guarantees are developed for settings with \(\rho _k = \rho \) ρ k = ρ as well as \(\eta _k = \eta \) η k = η . Preliminary numerics on a fused Lasso problem display promise.