Scholarlink

High-Performance Computing (HPC) systems plays a crucial role for executing large-scale computational workloads. However, users frequently encounter shortcomings in job success rate due to suboptimal node assignments, leading to prolonged execution times or incomplete jobs. To reduce this, we propose LBE (Look Before Execute), a machine learning-based method that foresees job execution time for different resource allocations before final job execution on the HPC system. LBE capitalizes on historical data of jobs and compute resources to train models that estimate runtime performance of the job on the allocated set of compute resources. This approach aims to reduce computational waste by early prediction of execution time on allocated resources, improve the user’s job success rates, and enhance the overall HPC system efficiency. Experimental results show that LBE can considerably helps to improve the user’s job executions on the HPC system by providing window to user for taking early decisions regarding the jobs, thus minimizing unwanted execution delays and failures.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Machine Learning Approach for Enhancing HPC User’s Job Efficiency Through Look Before Execution Method

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Om Jadhav,
  • Dharm Singh Jat,
  • Sabbi Vamshi Krishna

摘要

High-Performance Computing (HPC) systems plays a crucial role for executing large-scale computational workloads. However, users frequently encounter shortcomings in job success rate due to suboptimal node assignments, leading to prolonged execution times or incomplete jobs. To reduce this, we propose LBE (Look Before Execute), a machine learning-based method that foresees job execution time for different resource allocations before final job execution on the HPC system. LBE capitalizes on historical data of jobs and compute resources to train models that estimate runtime performance of the job on the allocated set of compute resources. This approach aims to reduce computational waste by early prediction of execution time on allocated resources, improve the user’s job success rates, and enhance the overall HPC system efficiency. Experimental results show that LBE can considerably helps to improve the user’s job executions on the HPC system by providing window to user for taking early decisions regarding the jobs, thus minimizing unwanted execution delays and failures.