Autonomous vehicles (AVs) have demonstrated significant potential in revolutionizing transportation, yet ensuring their safety and reliability remains a critical challenge, especially when exposed to dynamic and unpredictable environments. Real-world testing of an Autonomous Driving System (ADS) is both expensive and risky, making simulation-based testing a preferred approach. In this paper, we propose \(\textsc {Scenetic}\) , a Reinforcement Learning (RL)-based approach to generate plausible critical scenarios for testing ADSs in simulation environments. To capture the complexity of driving scenarios, \(\textsc {Scenetic}\) comprehensively represents the environment by both the internal states of an ADS under-test (e.g., the status of the ADS’s core components, speed, or acceleration) and the external states of the surrounding factors in the simulation environment (e.g., weather, traffic flow, or road condition). \(\textsc {Scenetic}\) trains the RL agent to configure the simulation environment that places the AV in dangerous situations and potentially leads it to collisions. We introduce a diverse set of actions that allows the RL agent to systematically configure both environmental conditions and traffic participants. Additionally, based on established safety requirements, we enforce heuristic constraints to promote the challenge and practical relevance of the generated test scenarios. \(\textsc {Scenetic}\) is evaluated on two popular simulation maps with four different road configurations. Our results show \(\textsc {Scenetic}\) ’s ability to outperform the state-of-the-art approach by generating 30% to 115% more collision scenarios. Compared to the baseline based on Random Search, \(\textsc {Scenetic}\) achieves up to 275% better performance. These results highlight the effectiveness of \(\textsc {Scenetic}\) in enhancing the safety testing of AVs through the generation of comprehensive and plausible critical scenarios.
