Deep Reinforcement Learning Based Load Balancing for Heterogeneous Traffic in Datacenter Networks

Modern high-speed datacenter networks (DCNs) employ multi-tree topologies to provide large bisection bandwidth. Load balancing is crucial for making full use of parallel equal-cost paths and ensuring high link utilization. In the past decades, a large number of heuristic load balancing mechanisms have been proposed to alleviate congestion. However, they cannot be resilient to the concurrent explosive growth and unpredictable dynamic traffic scenarios. Recently, deep reinforcement learning methods have become very powerful techniques for reacting to dynamically changing networks, but the existing learning-based load balancing schemes are agnostic to the heterogeneous traffic generated by diverse applications. Thus, the latency-sensitive short flows suffer from large tail delays due to the long pre-learning period. To address the above issues, we propose a deep reinforcement learning-based load balancing mechanism called DRLB. Specifically, DRLB differentiates heterogeneous traffic by using Deep Reinforcement Learning (DRL) in conjunction with the Distributed Distributional Deterministic Policy Gradients (D4PG) algorithm to make the optimal (re)routing for long flows while adopting the Weighted Cost Multipathing (WCMP) mechanism for short flows. The NS-3 experimental results show that, DRLB increases the throughput of long flows by up to 47% and reduces the flow completion time (FCT) of short flows by up to 58% compared to the state-of-the-art load balancing mechanisms.