A Novel Approach to Dynamic Computing Using Slurm
摘要
During operations of the Swiss Light Source (SLS), it was customary for the high-volume scientific beamlines to have statically dedicated resources assigned to them. While providing a simple and predicting approach to resources assignation, this systematically led either to resource starvation during peak times, or waste of idle compute cycles. The push for better resources utilization during the operations of the upgraded facility (SLS2), and a more dynamical assignation of them, led to explore a novel approach to cluster management using a workload manager such as Slurm. By combining different Slurm and Systemd functionalities, we architected a way to move computing resources dynamically between clusters that may have different rules and requirements (e.g. accesses to different file systems), and we called it flurm, as in flexible slurm. The advantage over normal resources assignation such as reservations or dedicated partitions resides in the customization possibilities (storage mountpoints, running services) that this architecture allows. This feature is offered directly to the final users, that upon certain conditions can stear resources without the admins intervention. In this paper we describe the situation during SLS operations, the novel architecture that was designed and its implementation, its current status and way forward.