DRL-Based Scheduling Scheme with Age of Information for Real-Time IoT Systems

With the rapid development of Mobile Edge Computing (MEC) and the Internet of Things (IoT) technology, real-time monitoring applications have become a part of our daily life. However, these applications rely on the timeliness of collecting environmental information. Therefore, we introduce the emerging metric of Information Age (AoI) to measure the freshness of information. In this article, Due to the simultaneous requests from multiple devices in an IoT system, we consider the data update sampled by the device can be computed by the device or offloaded directly to the destination for computing, jointly design offloading and scheduling policies in sequential time frames to minimize the average weighted sum of AoI and energy consumption. We first formulate the optimization problem as multi-stage non-linear integer programming (NLP) problem. Secondly, to reduce the computational complexity, we develop a learning-based algorithm based on emerging deep reinforcement learning (DRL) to reduce the dimensionality of state space and utilize a late experience storage method to train a heterogeneous deep neural networks (DNNs) synchronously during the training process. Meanwhile, one exploration policy is designed to obtain multiple candidate actions based on single real-number output of neural network. The proposed policy method provides higher diversity in the generated actions and the performance is near-optimal.