Proximal Policy Optimization(PPO) is a widely utilized Deep Reinforcement Learning algorithm in fields like robotics, manufacturing, and automated guided vehicles. However, models trained by the ordinary PPO continue to encounter challenges related to model instability and ineffective training, stemming from insufficient learning facilitated by neural networks. Recently, the Kolmogorov–Arnold Network (KAN) was proposed as a function approximation alternative to Multi-Layer Perceptrons (MLPs) because of the high accuracy of learning brought by fine-grained network structure. This paper has integrated KAN into the PPO framework, namely KAN-PPO. Through meticulously designed experiments conducted across OpenAI Gym environments, the KAN-PPO algorithm has demonstrated rapid convergence, stable training processes, and accelerated model learning with the same parameter volume and network layers as the original MLP-PPO. Furthermore, KAN-PPO exhibits effective learning, requiring lighter parameter volumes and network depth when tackling equivalent tasks. Consequently, the proposed KANPPO holds significant potential for application in addressing industrial challenges.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

KAN-PPO: A Fast Convergence and Stable Proximal Policy Optimization Powered by Kolmogorov–Arnold Network

  • Ruxin Xiao,
  • Jialu Sun,
  • Yuchen Wang,
  • Ziren Xiao,
  • Honghao Gao,
  • Muddesar Iqbal,
  • Peng Ren,
  • Cai Luo,
  • Xinheng Wang

摘要

Proximal Policy Optimization(PPO) is a widely utilized Deep Reinforcement Learning algorithm in fields like robotics, manufacturing, and automated guided vehicles. However, models trained by the ordinary PPO continue to encounter challenges related to model instability and ineffective training, stemming from insufficient learning facilitated by neural networks. Recently, the Kolmogorov–Arnold Network (KAN) was proposed as a function approximation alternative to Multi-Layer Perceptrons (MLPs) because of the high accuracy of learning brought by fine-grained network structure. This paper has integrated KAN into the PPO framework, namely KAN-PPO. Through meticulously designed experiments conducted across OpenAI Gym environments, the KAN-PPO algorithm has demonstrated rapid convergence, stable training processes, and accelerated model learning with the same parameter volume and network layers as the original MLP-PPO. Furthermore, KAN-PPO exhibits effective learning, requiring lighter parameter volumes and network depth when tackling equivalent tasks. Consequently, the proposed KANPPO holds significant potential for application in addressing industrial challenges.