Resource Elasticity for Scientific Platforms on HPC Infrastructure
摘要
Scientists expect HPC systems to support scalable jobs, deliver consistent time-to-solution, and offer flexibility in software and hardware configurations. Accommodating this combination of scalable jobs, reproducible performance, and custom environments is particularly challenging for HPC service providers operating large-scale, general-purpose supercomputers. The Alps infrastructure introduces a novel approach to address these challenges by using versatile software defined cluster (vCluster) technology to create isolated, customizable environments tailored to specific research communities. However, when defined as static resource sets, such partitions limit the ability to accommodate highly scalable jobs or respond dynamically to workload changes. To address this, we propose a solution that enables elastic resource sharing across scientific platforms within the management stack of the supercomputing system. By replaying job traces from a production supercomputer, we demonstrate that our approach can meet multiple quality-of-service (QoS) objectives while maintaining high overall resource utilization.