The phenomenon of deep stall refers to the situation where an aircraft, due to the presence of a stable point at high angles of attack, tends to remain in this state once it enters deep stall. This condition leads to a reduction in lift and control effectiveness, making it challenging to recover using conventional controls. This paper proposes a control method based on a reinforcement learning algorithm to achieve deep stall recovery. The state and action spaces are determined based on the aircraft motion equations. The reward function is designed considering flight characteristics, and the Proximal Policy Optimization algorithm is employed to train the controller for end-to-end deep stall recovery. Simulation results demonstrate that the proposed deep stall recovery method effectively achieves recovery and maintains stable aircraft states after recovery. Additionally, this paper considers constraints on the smoothness of controller outputs and examines the robustness of the controller to state noise.