In complex scenarios, rapid and accurate decision-making is crucial for autonomous vehicles. Deep reinforcement learning has attracted extensive attention due to its exceptional decision-making capabilities. However, existing deep reinforcement learning algorithms suffer from slow network convergence and suboptimal decision outcomes due to the vast state space and inadequate feature extraction capabilities of the representation models. To address this issue, we have integrated the Convolutional Block Attention Module (CBAM) with Long Short-Term Memory (LSTM) networks to develop the Memory Attention Convolutional Neural Networks (MACNN). This network serves as a feature representation model, capable of extracting environmental information that includes historical features from sequential bird’s-eye view inputs. We have integrated the MACNN with the Proximal Policy Optimization (PPO) network to form the MACNN-PPO algorithm. Simulation experiments conducted in the Carla simulator demonstrate that the MACNN-PPO algorithm achieves faster convergence speeds and higher convergence rewards during training. With the same number of training iterations, the MACNN-PPO maintains a higher driving speed and improves route completion rates in complex urban environments.