User-provided job walltime (runtime) estimates are often used in current job schedulers to optimize performance and predictability when scheduling parallel jobs on computing resources. Since the walltime estimate is usually the upper runtime limit after which a job is forcibly terminated, user-provided estimates are usually overestimated, inaccurate, and rather coarse-grained. In this situation, the performance of crucial scheduling policies (e.g., the ability to backfill jobs) can be degraded. In this paper, we present a large real-life workload trace from the Czech National Grid Infrastructure MetaCentrum that contains automatically generated runtime estimates that improve original (user-provided) job walltime estimates. We also present our experience with using an automated system to improve user-provided estimates in an actual HPC system. Last but not least, we provide insights into the nature of the workload and also share several tools used to develop, debug, and analyze our predictor, scheduling policies, and the workload itself.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Life HPC Workload Trace Featuring Refined Job Runtime Estimates

  • Dalibor Klusáček,
  • Václav Chlumský

摘要

User-provided job walltime (runtime) estimates are often used in current job schedulers to optimize performance and predictability when scheduling parallel jobs on computing resources. Since the walltime estimate is usually the upper runtime limit after which a job is forcibly terminated, user-provided estimates are usually overestimated, inaccurate, and rather coarse-grained. In this situation, the performance of crucial scheduling policies (e.g., the ability to backfill jobs) can be degraded. In this paper, we present a large real-life workload trace from the Czech National Grid Infrastructure MetaCentrum that contains automatically generated runtime estimates that improve original (user-provided) job walltime estimates. We also present our experience with using an automated system to improve user-provided estimates in an actual HPC system. Last but not least, we provide insights into the nature of the workload and also share several tools used to develop, debug, and analyze our predictor, scheduling policies, and the workload itself.