As global urbanization accelerates, municipal solid waste management has become a critical environmental challenge. Waste separation constitutes a fundamental strategy for achieving urban sustainability. Understanding the drivers of public participation in separation is critical importance to both research and practice. This study focuses on the core topic of residents’ waste separation behavior, and deeply explores the underlying mechanism of environmental regulation on residents’ waste separation behavior. Questionnaire survey (N = 408) was used to collect empirical data, and statistical methods such as hierarchical regression analysis were conducted for data. The results show that regulation strategy has a significant positive effect on residents’ waste separation behavior, indicating that the regulation intensity is positively correlated with the separation behavior. Furthermore, perception of policy effectiveness had a positive impact on the separation behavior. Finally, perceived policy effectiveness plays a mediating role in regulation promoting separation behavior. These findings provide important theoretical support for improving the waste separation policy system. These findings provide robust theoretical support for refining waste separation policies. We recommend that governments establish a comprehensive regulatory system that combines institutional enforcement with behavioral guidance, while simultaneously enhancing policy transparency to cultivate trust and sense of ownership.