<p>With the rapid development of smart manufacturing and IoT technologies, traditional rule-based automation faces significant challenges in multi-source data integration, real-time decision-making, and cross-device coordination. This study proposes a novel dynamic manufacturing environment framework based on a decentralized Multi-Agent System (MAS-DME) that uniquely combines multi-agent architectures with hierarchical reinforcement learning. MAS-DME consists of two core agents: (1) an Equipment Control Agent leveraging Double Deep Q-Network (Double DQN) for real-time equipment state monitoring and adaptive control, and (2) a Resource Allocation Agent employing hierarchical reinforcement learning for dynamic and flexible resource scheduling. To validate the proposed work, comparative experiments against current production control methods were conducted by evaluating several critical performance indicators, including production efficiency, resource utilization, equipment failure response time, and product quality. Comparative experiments against existing production control methods demonstrate that MAS-DME significantly enhances production efficiency, optimizes resource utilization, and substantially reduces equipment failure response times. This research not only advances the theoretical foundations of dynamic process optimization in smart manufacturing but also offers practical pathways for future applications, including intelligent supply chain management and adaptive industrial automation.</p>