<p>In intelligent stamping workshops, the characteristics of sudden production tasks, compact equipment layout, and frequent material flow path conflicts make the scheduling problem of Automated Guided Vehicles (AGVs) under dynamic constraints a critical bottleneck affecting production efficiency. Traditional Deep Q-Network (DQN) algorithms, while capable of optimizing decision-making strategies through experience replay mechanisms, struggle to adapt to the high-dimensional state space and sparse reward problems in workshop environments due to their single-value network architecture and fixed exploration strategies. This often leads to local optima and low learning efficiency. To address these issues, this study proposes a multi-guided exploration algorithm based on Double Deep Q-Network (DDQN). By introducing a dynamic exploration weight mechanism and priority-based experience replay technology, the algorithm enhances exploration efficiency in constrained spaces. Specifically, a Markov Decision Process (MDP)-based AGV scheduling model is first established. Then, an adaptive environment exploration parameterization model based on deep learning is developed to solve material scheduling problems in real workshop environments. Using a stamping manufacturing workshop as a case study, experiments comparing the improved DDQN, improved DQN, and traditional DQN algorithms demonstrate the feasibility and effectiveness of the proposed method. Simulation results show that the improved DDQN algorithm significantly enhances task completion rates and training convergence speed, proving its robustness in highly dynamic and multi-constrained industrial scenarios. This study provides a scheduling decision framework for intelligent manufacturing systems that balances real-time performance and interpretability, offering practical guidance for the flexible upgrading of discrete manufacturing industries.</p>