Scholarlink

<p>In this paper, we address the issue of fairness in preference-based reinforcement learning (PbRL) in the presence of multiple objectives. The main objective is to design control policies that can optimize multiple objectives while treating each objective fairly. Toward this objective, we design a new fairness-induced preference-based reinforcement learning or FPbRL. The main idea of FPbRL is to learn vector reward functions associated with multiple objectives via new <i>welfare-based</i> preferences rather than <i>reward-based</i> preferences in PbRL, coupled with policy learning via maximizing a welfare function. Finally, we conduct experiments to compare FPbRL and other relevant techniques in different real-world environments, considering synthetic human teacher preferences with various imperfection and irrationality factors, including myopic behavior, deviations, and perturbed feedback. Our study shows that the proposed FPbRL approach strikes a balance between efficiency and equity, effectively learning policies that are both efficient and impartial even in the presence of imperfect preferences.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Learning Fair Policies in Multi-objective Preference-Based Reinforcement Learning

  • 期刊论文

  • 发表日期：

    2026年1月13日

查看全文
  • Umer Siddique,
  • Abhinav Sinha,
  • Yongcan Cao

摘要

In this paper, we address the issue of fairness in preference-based reinforcement learning (PbRL) in the presence of multiple objectives. The main objective is to design control policies that can optimize multiple objectives while treating each objective fairly. Toward this objective, we design a new fairness-induced preference-based reinforcement learning or FPbRL. The main idea of FPbRL is to learn vector reward functions associated with multiple objectives via new welfare-based preferences rather than reward-based preferences in PbRL, coupled with policy learning via maximizing a welfare function. Finally, we conduct experiments to compare FPbRL and other relevant techniques in different real-world environments, considering synthetic human teacher preferences with various imperfection and irrationality factors, including myopic behavior, deviations, and perturbed feedback. Our study shows that the proposed FPbRL approach strikes a balance between efficiency and equity, effectively learning policies that are both efficient and impartial even in the presence of imperfect preferences.