Scholarlink

<p>Dynamic resource management enables supercomputing applications to change resource allocations at runtime. This capability promises significant improvements in application efficiency and overall supercomputer utilization. However, adoption is limited by insufficient support in resource managers and programming environments. Furthermore, developing resource-flexible applications introduces significantly higher programming complexity than their static counterparts. While MPI extensions have been proposed for resource flexibility, significant programmability challenges persist. The “Dynamic Processes with PSets (DPP)” design principles define programming model agnostic abstractions for dynamic resource control, and have been prototypically implemented by extending Open&#xa0;MPI and OpenPMIx (termed <span>MPI-DPP</span>). <span>MPI-DPP</span> enables fine-grained process management but relies on low-level message-passing, complicating implementation of dynamic and irregular workloads. Asynchronous Many-Task (AMT) programming offers a compelling alternative. AMT splits computations into fine-grained tasks dynamically scheduled by the runtime system, enabling load balancing and responsiveness to resource changes. Although resource-flexible AMTs remain rare, <span>GLB</span> is a notable exception, offering automatic load balancing and dynamic resource capabilities. However, <span>GLB</span> is built on “APGAS for Java”, which is uncommon in HPC. We present <span>DPP-GLB</span>, a C++ AMT runtime that integrates <span>GLB</span> ’s high-level task abstraction and load balancing with the resource control capabilities of <span>MPI-DPP</span>. We evaluate <span>DPP-GLB</span>, <span>GLB</span>, and <span>MPI-DPP</span> on SuperMUC-NG, analyzing both programming complexity and runtime performance. Results show that <span>GLB</span> is easy to use, featuring built-in load balancing and resource flexibility. <span>MPI-DPP</span> offers superior performance for node changes, albeit at the cost of increased programming complexity. <span>DPP-GLB</span> achieves a balance of low programming complexity and efficient, scalable dynamic resource support.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Toward Dynamic Resource Management: An Asynchronous Many-Task (AMT) Runtime System leveraging Dynamic Processes with PSets (DPP)

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月5日

查看全文
  • Jonas Posner,
  • Tim Ellersiek,
  • Nick Bietendorf,
  • Dominik Huber,
  • Martin Schreiber,
  • Martin Schulz

摘要

Dynamic resource management enables supercomputing applications to change resource allocations at runtime. This capability promises significant improvements in application efficiency and overall supercomputer utilization. However, adoption is limited by insufficient support in resource managers and programming environments. Furthermore, developing resource-flexible applications introduces significantly higher programming complexity than their static counterparts. While MPI extensions have been proposed for resource flexibility, significant programmability challenges persist. The “Dynamic Processes with PSets (DPP)” design principles define programming model agnostic abstractions for dynamic resource control, and have been prototypically implemented by extending Open MPI and OpenPMIx (termed MPI-DPP). MPI-DPP enables fine-grained process management but relies on low-level message-passing, complicating implementation of dynamic and irregular workloads. Asynchronous Many-Task (AMT) programming offers a compelling alternative. AMT splits computations into fine-grained tasks dynamically scheduled by the runtime system, enabling load balancing and responsiveness to resource changes. Although resource-flexible AMTs remain rare, GLB is a notable exception, offering automatic load balancing and dynamic resource capabilities. However, GLB is built on “APGAS for Java”, which is uncommon in HPC. We present DPP-GLB, a C++ AMT runtime that integrates GLB ’s high-level task abstraction and load balancing with the resource control capabilities of MPI-DPP. We evaluate DPP-GLB, GLB, and MPI-DPP on SuperMUC-NG, analyzing both programming complexity and runtime performance. Results show that GLB is easy to use, featuring built-in load balancing and resource flexibility. MPI-DPP offers superior performance for node changes, albeit at the cost of increased programming complexity. DPP-GLB achieves a balance of low programming complexity and efficient, scalable dynamic resource support.