Deep Q-Learning-Based Adaptive Multimedia Streaming in Vehicular Edge Intelligence

In this chapter, we present an architecture for Adaptive-BitRate (ABR)-based multimedia streaming in heterogeneous IoV, where each multimedia file is segmented into multiple chunks, which are encoded with different bitrate levels. Then, we formulate a Joint Resource Optimization (JRO) problem by synthesizing heterogeneous edge cache and communication resource constraints, aiming at achieving both smooth playback and high-quality service by optimizing chunk placement and transmission. For chunk placement, a multi-armed bandit (MAB) algorithm is proposed for online scheduling with low overhead but slow convergence. Further, a deep Q-learning algorithm is proposed to improve cache reward and speed up convergence by using replay memory. For chunk transmission, we design an Adaptive-Quality-based Chunk Selection (AQCS) algorithm that determines bandwidth allocation and quality level based on a benefit function, which considers quality level, available playback time, and freezing delay. Lastly, we build the simulation model and give comprehensive performance evaluation, which demonstrates the superiority of proposed algorithms.