Design and Analysis of Goal-Oriented Reward System on Deep Reinforcement Learning SDN Migration Framework
摘要
The adoption of Software-Defined Networking (SDN) faces challenges due to existing infrastructure investments and migration costs, leading to the emergence of incremental hybrid SDN deployment. A critical challenge in hybrid SDN implementation is determining the effective node migration sequence while simultaneously considering immediate performance gains and effective load balancing. This paper investigates the impact of different reward systems in a Deep Reinforcement Learning (DRL)-driven SDN migration framework under dynamic network conditions. We propose three distinct reward functions focusing on local goal optimization, global goal optimization, and a combined approach integrating both goals. Experiments conducted on Abilene and GEANT network topologies demonstrate that the combined approach achieves superior results, requiring only 4 and 12 nodes respectively to attain minimum median Maximum Link Utilization (MLU), compared to local goal-oriented approach requiring 5 and 14 nodes. The combined approach maintains efficient convergence times (143 episodes for Abilene and 233 episodes for GEANT), comparable to single-goal reward functions. Our research contributes by: firstly, proposing three distinct reward functions for SDN migration, followed by demonstrating the combined approach’s superiority in achieving both objectives, and finally revealing that local and global goals can effectively complement each other within a well-designed reward system.