In the rapidly evolving landscape of artificial intelligence, efficient scheduling of computational resources has become pivotal for achieving high efficiency and throughput in machine learning training tasks. This study focuses on optimizing resource scheduling strategies for AI training tasks in heterogeneous computing environments, proposing a dynamic resource adjustment algorithm based on task performance metrics. The algorithm monitors key indicators such as processing speed and resource utilization in real time, utilizing predictive models to forecast task execution paths and dynamically adjust resource allocation to enhance computational resource utilization efficiency. Experimental validation conducted on neural network training tasks of various scales using the TensorFlow framework on heterogeneous platforms comprising CPUs, GPUs, and TPUs shows promising results. Compared to traditional methods, the proposed resource scheduling strategy demonstrates an average improvement of 17.7% in training efficiency and 20.5% in throughput, while reducing idle resource usage and waste, thereby significantly enhancing task performance. Furthermore, to accommodate diverse AI training requirements, the strategy considers task priorities and resource budget constraints, ensuring cost-effectiveness alongside performance enhancement.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Research on Optimizing Resource Scheduling Strategies Based on AI Training Task Metrics

  • Ruifeng Guo,
  • Chuanchen Wang,
  • Hongliang Wang,
  • ChangYi Deng,
  • Wei Wang

摘要

In the rapidly evolving landscape of artificial intelligence, efficient scheduling of computational resources has become pivotal for achieving high efficiency and throughput in machine learning training tasks. This study focuses on optimizing resource scheduling strategies for AI training tasks in heterogeneous computing environments, proposing a dynamic resource adjustment algorithm based on task performance metrics. The algorithm monitors key indicators such as processing speed and resource utilization in real time, utilizing predictive models to forecast task execution paths and dynamically adjust resource allocation to enhance computational resource utilization efficiency. Experimental validation conducted on neural network training tasks of various scales using the TensorFlow framework on heterogeneous platforms comprising CPUs, GPUs, and TPUs shows promising results. Compared to traditional methods, the proposed resource scheduling strategy demonstrates an average improvement of 17.7% in training efficiency and 20.5% in throughput, while reducing idle resource usage and waste, thereby significantly enhancing task performance. Furthermore, to accommodate diverse AI training requirements, the strategy considers task priorities and resource budget constraints, ensuring cost-effectiveness alongside performance enhancement.