Autonomous Underwater Vehicles (AUVs) are often required to navigate in complex marine environments during mission execution, facing challenges such as obstacles, terrain variations, and dynamic targets. Traditional path tracking methods struggle to handle these complex situations effectively. Therefore, planning safe and efficient paths in dynamically changing environments has become a critical research issue. This paper addresses the path tracking control problem of underactuated AUVs in three-dimensional space by proposing a path tracking method that combines Long Short-Term Memory (LSTM) networks with the Proximal Policy Optimization (PPO) algorithm. By introducing the LSTM network to extract environmental features and combining it with the strategy optimization ability of PPO, a path tracking framework that adapts to dynamic marine environments is designed. The LSTM network architecture is used to reconstruct the strategy and evaluation networks of the PPO algorithm, and a multi-objective integrated reward mechanism is designed, including distance-guided rewards, collision risk grading rewards, and task state rewards. Simulation results show that for two-dimensional path tracking tasks, the proposed method reduces the average error by 54.8% and 69.0% compared to the LSTM-DDPG algorithm and Cascade-Backstepping Algorithm, respectively. In three-dimensional path tracking tasks, the average error is reduced by 16.4% compared to the LSTM-DDPG algorithm and by 36.3% compared to traditional control algorithms. Additionally, the reward value convergence curve during training indicates faster convergence and stronger robustness. Overall, the simulation results demonstrate that the proposed LSTM-PPO algorithm can effectively achieve path tracking and obstacle avoidance in underwater environments, with good stability and adaptability.