Multi-objective reinforcement learning for recommender systems: a comprehensive survey of methods, challenges, and future directions
摘要
Most traditional recommender systems (RSs) prioritize accuracy-based metrics, often favoring popular items while neglecting novelty, diversity, and user engagement. However, real-world recommendation scenarios involve multiple conflicting objectives, requiring advanced optimization techniques. Multi-Objective Optimization (MOO) has been explored in RSs using collaborative filtering and evolutionary algorithms, but these approaches suffer from cold-start issues, data sparsity, and high computational complexity. Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) offer promising alternatives by dynamically learning optimal policies. However, single-objective DRL struggles to balance accuracy with non-accuracy metrics, making it less effective for real-world trade-offs. This has led to the rise of Multi-Objective Reinforcement Learning (MORL), which optimizes conflicting objectives using Pareto-based optimization, scalarization, and policy gradient adaptations. Unlike prior surveys on RL-based RSs, this review specifically examines MORL techniques, providing a structured taxonomy, evaluation of optimization strategies, and analysis of their impact on personalization, diversity, fairness, and engagement. We also highlight unresolved challenges, including scalability, sample efficiency, computational complexity, and real-time applicability. Finally, we propose future research directions to enhance MORL’s scalability, improve policy generalization, and integrate more efficient multi-objective learning techniques for large-scale recommendation systems.