A Dual-Layer Network Deep Reinforcement Learning Algorithm for Multi-objective Signal Temporal Logic Tasks

The path planning strategies of multi-objective tasks for autonomous agents are applied more and more widely in the areas of resource exploration, factory transportation, post-disaster relief, etc. These tasks typically require access to all target points within a limited time and can be formulated by signal temporal logic (STL) formulas with multiple logic AND. This paper presents a dual-layer network deep reinforcement learning algorithm to address the decision problem of the multi-objective STL tasks. First, the multi-objective optimization problem is represented by iterable UNTIL operators, and each sub-task is encoded into an one-hot vector. In the control layer, one-hot vector is updated according to the ongoing sub-task. Based on the coupling input of agent states and one-hot vector, the neural network controller is trained to complete the sub-tasks in the given order strictly. In the order layer, robust-based weighted entropy is assigned according to the agent trajectory robust value of each sub-task. By maximizing the entropy, the order priority of access to the target points is updated periodicity and is passed into the control layer. Simulation and comparison with the traditional method show that our proposed dual-layer network algorithm has the reliability for strictly completing sub-tasks of multi-objective task reformed by UNTIL operators and the effectiveness of the access order planning for multi-objective tasks with improved real-time performance.