Software Defined Wide Area Network (SD-WAN) offers an overlay solution for businesses to interconnect several remote branches to a common enterprise cloud. The SD-WAN overlay is built over several WAN transport connections (e.g., MPLS, 4G/5G or Internet services). The network operator then defines traffic engineering policies to balance the traffic load over the different WAN underlays. In this work we consider streams of IP packets, as in open-flow, which we call flows. We explore per-flow load-balancing which offers a more versatile and fine grained policy definition to the network operator. We formulate the load-balancing problem in a control theoretic framework. We compare two approaches to solve the problem: an optimal closed-form solution to the Model Predictive Control (MPC) formulation and a Deep Reinforcement Learning (DRL) Proximal Policy Optimization (PPO) based algorithm. We show that the approximate DRL algorithm reaches the MPC solution with less than 5 ‰ error on the load balancing ratio for the Poisson traffic case we considered and that it can address a large class of mixed traffic through Machine Learning generalisation.