Scholarlink

<p>We study the adaptive infinite-horizon discounted control problem for Piecewise Deterministic Markov Processes (PDMPs) using a Nonstationary Value Iteration (NVI) scheme. PDMPs, as introduced by Davis (Markov models and optimization, monographs on statistics and applied probability, Chapman and Hall, London, 1993) evolve deterministically between random jumps whose jump rate <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\lambda \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>λ</mi> </math></EquationSource> </InlineEquation>, transition measure&#xa0;<i>Q</i>, and cost&#xa0;<i>C</i> depend on an unknown parameter&#xa0;<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\beta ^*\)</EquationSource> <EquationSource Format="MATHML"><math> <msup> <mi>β</mi> <mo>∗</mo> </msup> </math></EquationSource> </InlineEquation>. The proposed NVI algorithm recursively updates the value function using current parameter estimates, enabling online implementation. We show that, for any sequence of strongly consistent estimators&#xa0;<InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\{\beta ^*_n\}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mo stretchy="false">{</mo> <msubsup> <mi>β</mi> <mi>n</mi> <mo>∗</mo> </msubsup> <mo stretchy="false">}</mo> </mrow> </math></EquationSource> </InlineEquation> converging almost surely to&#xa0;<InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\beta ^*\)</EquationSource> <EquationSource Format="MATHML"><math> <msup> <mi>β</mi> <mo>∗</mo> </msup> </math></EquationSource> </InlineEquation>, the resulting policy is asymptotically optimal under the discounted criterion.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Adaptive nonstationary value iteration for discounted control of Piecewise Deterministic Markov Processes

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月30日

查看全文
  • O. L. V. Costa,
  • F. Dufour,
  • A. Genadot

摘要

We study the adaptive infinite-horizon discounted control problem for Piecewise Deterministic Markov Processes (PDMPs) using a Nonstationary Value Iteration (NVI) scheme. PDMPs, as introduced by Davis (Markov models and optimization, monographs on statistics and applied probability, Chapman and Hall, London, 1993) evolve deterministically between random jumps whose jump rate \(\lambda \) λ , transition measure Q, and cost C depend on an unknown parameter  \(\beta ^*\) β ∗ . The proposed NVI algorithm recursively updates the value function using current parameter estimates, enabling online implementation. We show that, for any sequence of strongly consistent estimators  \(\{\beta ^*_n\}\) { β n ∗ } converging almost surely to  \(\beta ^*\) β ∗ , the resulting policy is asymptotically optimal under the discounted criterion.