Run-Time Assured Reinforcement Learning for Safe Spacecraft Rendezvous with Obstacle Avoidance

Autonomous spacecraft rendezvous poses significant challenges in increasingly complex space missions. Recently, Reinforcement Learning (RL) has proven effective in the domain of spacecraft rendezvous, owing to its high performance in complex continuous control tasks and low online storage and computation cost. However, the lack of safety guarantees during the learning process restricts the application of RL to safety-critical control systems within real-world environments. To mitigate this challenge, we introduce a safe reinforcement learning framework with optimization-based Run-time Assurance (RTA) for spacecraft rendezvous, where the safety-critical constraints are enforced by Control Barrier Functions (CBFs). First, we formulate a discrete-time CBF to implement dynamic obstacle avoidance within uncertain environments, concurrently accounting for soft constraints of spacecraft including velocity, time, and fuel. Furthermore, we investigate the effect of RTA on reinforcement learning training performance in terms of training efficiency, satisfaction of safety constraints, control efficiency, task efficiency, and duration of training. Additionally, we evaluate our method through a spacecraft docking experiment conducted within a two-dimensional relative motion reference frame during proximity operations. Simulation and expanded test demonstrate the effectiveness of the proposed method, while our comprehensive framework employs RL algorithms for acquiring high-performance controllers and utilizes CBF-based controllers to guarantee safety.