HiRM: Hierarchical resource management for earth system models on many-core clusters

Executing the resource-consuming parallel model, Earth System Models (ESMs), on the many-core CPU clusters can effectively improve the performance due to the clusters’ higher system scalability and lower intra-node communication cost. However, the extra cost still exists in inter-node neighborhood communication task inner Ocean component. Meanwhile, the overall execution time is vulnerable to the resource allocation of the individual components and the global layout. To address the problem, we propose a hierarchical resource management method, including a component-level task partitioning method to achieve better simulation performance on given resources and a global-level coordinated process scheduling algorithm to allocate resources to each component. We focus on the task partitioning for the neighborhood communication in Ocean component, which generates non-negligible inter-node neighborhood communication cost in large-scale parallelism. We ported one of ESMs, community earth system model (CESM), to the ARM-based many-core Kunpeng platform to evaluate the performance of the proposed methods. Extensive experiments have been conducted on many-core and multi-core platforms. Results illustrate that the optimized ESMs instance on an many-core platform has significant performance improvement, up to 71.62% overall. Other detailed performance analysis are also presented to demonstrate our contributions.