Scholarlink

Role-playing agents are becoming increasingly crucial in interactive AI systems, yet current approaches mainly rely on prompt engineering or fine-tuning to condition agents on predefined character profiles. These methods typically encounter challenges in maintaining fine-grained attribute alignment and handling low-knowledge scenarios, often leading to inconsistent role expressions across diverse contexts and when background information is limited. To overcome these limitations, we propose Attribute-Aligned Policy Optimization (AAPO), a reinforcement learning framework that integrates Direct Preference Optimization (DPO) or Proximal Policy Optimization (PPO) to explicitly strengthen the alignment between agent behaviors and role attributes, enabling faithful maintenance of role personality, language style, and background traits. Experiments demonstrate that AAPO significantly outperforms prompt-based and fine-tuning baselines in both LLM automated evaluations and human evaluations. Notably, AAPO exhibits consistent superiority in high-knowledge and low-knowledge settings, ensuring stable attribute adherence even under limited role information.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reinforcement Learning-Based Attribute Alignment for Role-Playing of LLMs

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Jiayang Fan,
  • Ting Bai

摘要

Role-playing agents are becoming increasingly crucial in interactive AI systems, yet current approaches mainly rely on prompt engineering or fine-tuning to condition agents on predefined character profiles. These methods typically encounter challenges in maintaining fine-grained attribute alignment and handling low-knowledge scenarios, often leading to inconsistent role expressions across diverse contexts and when background information is limited. To overcome these limitations, we propose Attribute-Aligned Policy Optimization (AAPO), a reinforcement learning framework that integrates Direct Preference Optimization (DPO) or Proximal Policy Optimization (PPO) to explicitly strengthen the alignment between agent behaviors and role attributes, enabling faithful maintenance of role personality, language style, and background traits. Experiments demonstrate that AAPO significantly outperforms prompt-based and fine-tuning baselines in both LLM automated evaluations and human evaluations. Notably, AAPO exhibits consistent superiority in high-knowledge and low-knowledge settings, ensuring stable attribute adherence even under limited role information.