QoS-Aware Cross-Domain Routing in SDN: A Comparative Study Between Competitive and Cooperative MARL Approaches

In distributed Software-Defined Networks (SDN), using multiple controllers brings many benefits but raises new challenges such as scalability and reliability. Mainly, challenges such as the QoS satisfaction and the network load balancing are still open area of research. Obviously, addressing these challenges will impact deeply the efficiency of the routing process. Recently, Multi-Agents Reinforcement Learning (MARL) approaches have attracted the attention of researchers to address efficiently the QoS-aware routing issue. Agents in MARL can either compete or cooperate in order to achieve the main objective. Making a decision on using either cooperative or competitive MARL is still a challenging issue due to the pros and cons of each approach. This paper aims at designing a new approach which achieves dynamic QoS handling and efficient inter-domain resources management in large-scale SDN networks. A Muti-Agents Reinforcement Learning (MARL)-based routing mechanism which uses a distributed DQN-based learning approach is suggested in order to achieve efficient QoS-aware routing and balanced inter-domain resources consumption. Simulations are conducted to evaluate the proposed approach in case where agents act competitively and when agents cooperate. Results show that competitive agents outperform cooperative agents for intelligent routing, in terms of load balancing and QoS satisfaction, regarding three metrics which are the throughput, the packet loss rate and the jitter.