Scheduling in Time-Sensitive Networks Using Deep Reinforcement Learning

Ethernet and wireless-based technologies play an important role in the Internet of Medical Things. The new healthcare scenarios require a deterministic communication system with real-time capabilities. In remote surgery, for example, the video latency is limited to 200 ms. In addition to latency, packet loss above a certain threshold can have a serious impact on the surgery. The majority of IP traffic today is served as best effort. Although assigning priority classes to flows can help, it is insufficient. A typical solution is to use high-speed media layers such as EtherCAT. However, these approaches do not integrate well with Ethernet. In this context, IEEE Time-Sensitive Networking (TSN) standards are gaining traction as a key enabler of real-time capabilities. TSN enables real-time communication by providing mechanisms for controlling latency, jitter, and packet loss. The purpose of this paper is to identify a scheduling strategy to minimize the violation of an end-to-end delay requirement while working with mixed-priority traffic, including time-critical ones. The majority of related work focus on static scheduling and cannot be learned continuously. This paper employs a deep reinforcement learning technique to enable dynamics and incremental learning. The scheduling problem is modeled as a Markov decision process, in which constraints are mapped to Rewards. In the proposed solution, learning converges in a few epochs and the results provide a feasible schedule. The schedule is then evaluated using simulation, which demonstrates that the deterministic requirements are met.