Transmission scheduling of P2P real-time communication based on restless multi-armed bandit

In this paper, we consider a transmission scheduling problem of the point-to-point (P2P) real-time communication over fading channel. Specifically, we assume each transmission task with a strict time delay arrives into the system randomly, we consider the transmission scheduling strategy under the time delay constraint of the communication system so as to maximize the total discounted expected reward of the system. The communication model is formulated in the framework of the Markov decision process (MDP), and since the curse of dimensionality of the proposed MDP architecture, the transmission scheduling problem is analyzed based on the restless multi-armed bandit process. We show the existence of the Whittle index of the P2P transmission communication, and based on which, the closed solution of Whittle index based transmission scheduling strategy is obtained. Finally, the numerical results are given to verify the availability of the Whittle index based transmission scheduling algorithm, and it is also shown that our proposed transmission scheduling algorithm is with low computational complexity, and it reduces the computational time greatly in comparison to the MDP method.