Decomposed Multi-objective Method Based on Q-Learning for Solving Multi-objective Combinatorial Optimization Problem

Neural combinatorial optimization has emerged as a promising technique for combinatorial optimization problems. However, the high representation of deep learning inevitably requires a lot of training overhead and computing resources, especially in large-scale decision-making and multi-objective scenarios. This paper first provides a simple but efficient combinatorial optimization method that uses a traditional reinforcement learning (RL) paradigm to balance the computational cost and performance. We decompose the multi-objective problem into multiple scalar subproblems and only use the improved Q-learning for the sequential optimization of these subproblems. Our method employs the Temporal-Difference (TD) update strategy and provides a shared Q-table for all subproblems. The TD update strategy speeds up the optimization by learning while making decisions. The shared Q-table devotes a high-quality starting point to generate excellent solutions quickly for each subproblem. Both strategies promote the effectiveness and efficiency of the proposed method. After new solutions are generated, a selection operator keeps the historical optimal solution for each subproblem. We apply our method to various multi-objective traveling salesman problems involving up to 10 objectives and 200 decisions. Experiments demonstrate that only simple RL achieved comparable performance to state-of-the-art approaches.