As AI-driven test script generation continues to evolve, automated test script generation has emerged as a key objective in software testing. However, effective evaluation of AI-generated test scripts for Graphical-user-interface (GUI) requires a practically relevant and fully automatable benchmark, since existing benchmarks often lack real-world applicability or reproducibility essential for the fairly compare evolving AI models. In this paper, we introduce a comprehensive benchmarking framework designed for UI testing of complex IT systems. Our framework comprises an immutable, containerized system under test (SUT) with frozen microservices and UI, instrumented to measure both backend white-box coverage and frontend UI interaction path similarity. A handcrafted reference test suite comprising scenarios difficulty levels by experts serves as the gold standard. The framework supports quantitative evaluation through automated metrics, including the extent of manual refinement required to make large language model (LLM) generated scripts executable and qualitative assessment of attributes like readability and maintainability. A case study evaluating multiple LLMs demonstrates the effectiveness of the benchmark in highlighting both strengths and limitations of current AI-facilitation approaches. This reproducible evaluation environment provides decision makers and tool developers with valuable insights for advancing AI-based testing strategies.