Scholarlink

<p>Agent behavior is shaped by latent decision parameters that govern how rewards are interpreted and traded off over time. A key example is the <i>discount factor</i>, which encodes time preference. Mis-specifying the discount factor can confound reward-centric behavioral models (e.g., inverse RL), motivating the need to infer time preference directly from behavior. This paper presents methods for <i>discount factor elicitation</i> in finite-state Markov Decision Processes via policy observations and controlled reward modifications. First, we introduce an algorithm that bounds the set of discount factors consistent with an agent’s observed (near-)optimal policy, and show how observations across heterogeneous reward settings progressively tighten these bounds. Building on this result, we propose an <i>active elicitation framework</i> in which an ego agent strategically adjusts rewards (with fixed dynamics) to refine its estimate of another agent’s discount factor. Through case studies, we demonstrate that active elicitation accelerates interval refinement relative to passive observation and enables <i>targeted exploration</i> in strategic multi-agent settings. Overall, our results establish reward modification as a principled mechanism for eliciting discount factors and improving behavioral modeling, prediction, and control.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Active discount factor elicitation via reward modification

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月1日

查看全文
  • Shadi Tasdighi Kalat,
  • Sriram Sankaranarayanan,
  • Ashutosh Trivedi

摘要

Agent behavior is shaped by latent decision parameters that govern how rewards are interpreted and traded off over time. A key example is the discount factor, which encodes time preference. Mis-specifying the discount factor can confound reward-centric behavioral models (e.g., inverse RL), motivating the need to infer time preference directly from behavior. This paper presents methods for discount factor elicitation in finite-state Markov Decision Processes via policy observations and controlled reward modifications. First, we introduce an algorithm that bounds the set of discount factors consistent with an agent’s observed (near-)optimal policy, and show how observations across heterogeneous reward settings progressively tighten these bounds. Building on this result, we propose an active elicitation framework in which an ego agent strategically adjusts rewards (with fixed dynamics) to refine its estimate of another agent’s discount factor. Through case studies, we demonstrate that active elicitation accelerates interval refinement relative to passive observation and enables targeted exploration in strategic multi-agent settings. Overall, our results establish reward modification as a principled mechanism for eliciting discount factors and improving behavioral modeling, prediction, and control.