A low-thrust maintenance approach for planar orbits in the Perturbed Circular Restricted Three-Body Problem via reinforcement learning

This paper introduces an approach to design an orbital maintenance strategy for periodic Lyapunov orbits within the Perturbed Circular Restricted Three-Body Problem (P-CRTBP) considering oblateness and photogravitational perturbations. When a satellite deviates from its periodic path, both its position and velocity vectors change simultaneously, requiring a small low-thrust maneuver to restore it to its periodic orbit. This low-thrust trajectory is treated as a Two-Point Boundary Value Problem (TPBVP), which demands a specific solution method. Previous studies have employed numerical optimal control techniques like nonlinear programming (NLP). However, these methods may involve complex mathematical implementation. To address these challenges, this paper proposes an approach using reinforcement learning (RL) to design orbital maintenance maneuvers. The proposed method involves four main steps. First the perturbed periodic orbits are obtained by employing Poincaré map and an orbital correction algorithm. This algorithm stands out from previous orbital correction algorithms by enabling the propagation of P-CRTBP family orbits around the Lagrangian points using just one initial guess derived from the Poincaré map. Second, the time instant at which the deviation from periodicity exceeds a prescribed numerical threshold is identified, together with the corresponding state vector. This identification is based on a sensitivity analysis to establish criteria for non-periodic behavior. A key innovation of this approach is selecting the state vector to which the satellite must return to maintain its periodic orbit as the spacecraft is guided back to the matching point of the first orbital period, identified in the third step of the procedure. This choice enables the satellite to resume its periodic motion. This suggestion is justified because, according to the adopted criteria, the distance between the state at which the deviation exceeds the numerical threshold and the corresponding point in the first orbital period of the Earth–Sun system is sufficiently small. A small thrust applied over a very short period would be sufficient to address this discrepancy. In the final step, the thruster components and their duration of working time are nominated as design variables. Two objective functions are then formulated and minimized using RL to determine these design variables. The first objective function is guaranteed that the state vector is converted from the first non-periodic moment state vector to the desired state vector by applying thruster components. Since fuel efficiency is crucial in space missions, the second objective function focuses on minimizing fuel consumption. These two objective functions are combined to form a reward function, and maximizing this reward function effectively leads to the minimization of the objective functions. The method was applied to the perturbed Sun–Earth system, and the simulation showed that the proposed approach could be effective for orbital maintenance missions.