<p>Tandem cold rolling is a complex manufacturing procedure that requires strict control over product size accuracy and production stability. Various interferences and uncertainties, such as lubrication status and equipment, can affect the unsteady rolling process. Under certain operating conditions, model-based methods may not have sufficient control capabilities, resulting in significant shear losses. Especially during acceleration and deceleration, the friction state between the rollers and the strip constantly changes as the speed of the rolling mill changes, which dynamically affects the rolling force and forward sliding. Due to these parameter changes, the proportional integral (PI) controller may be unable to handle control tasks effectively. This article presents a model-free control algorithm that uses a multi-agent strategy and constraint reinforcement learning approach. Multiple controllers are trained under a centralized training and decentralized execution framework without mathematical models. To maintain flexibility in control strategies for each stand under varying observation information, agents can coordinate and cooperate using the same value function network to achieve coupling control of multiple stands. This paper also explores constrained roller speed actions to achieve a smooth and secure acceleration and deceleration process in compliance with safety standards. The simulation experiment results demonstrate that this method provides greater control accuracy and improved stationarity for thickness and tension compared to traditional PI during acceleration and deceleration processes.</p>