The quadruped robot still faces many challenges in achieving efficient, stable, and adaptive control strategies. Although Reinforcement Learning (RL) has shown great performance in continuous control tasks, its application on legged robots is constrained by partial observability and temporal dependency issues. Existing methods use the Proximal Policy Optimization (PPO) based Actor-Critic structure, which offers advantages such as stability and ease of parallel training. However, the traditionally used Multi-Layer Perceptron (MLP) network lacks memory capabilities and struggles to capture historical motion information of the robot, leading to myopic policies that affect overall performance in complex tasks. To address this, this paper proposes an improved PPO-RNN structure, which integrates a Recurrent Neural Network (RNN) into the Actor-Critic framework to enhance the neural network’s ability to model historical observation information. By replacing the original MLP module with an RNN module that can build temporal information flow, the proposed method can better handle the temporal dependency problem in Partially Observable Markov Decision Processes (POMDP) and improve the stability and robustness of the policy. Through comparative experiments, we demonstrate the significant advantages of PPO-RNN in terms of convergence speed, cumulative rewards, and motion trajectory smoothness. The experimental results show that, compared to the traditional PPO-MLP, the proposed method improves the motion control performance of the quadruped robot in complex environments.