A High-Performance Collective I/O Framework Leveraging Node-Local Persistent Memory

Collective I/Os are widely used to transform small, non-contiguous accesses into large, contiguous accesses for parallel I/O optimization. The existing collective I/O techniques were proposed with the assumption that computer memory is volatile. However, their ability is limited by the size of collective I/O buffers and communication overhead. In this paper, we propose PMIO, a novel collective I/O framework that employs node-local persistent memory on compute nodes for I/O optimization of HPC applications. First, it uses a log-structured buffer to achieve a high bandwidth of persistent memory and enforce crash consistency, allowing us to increase buffer size. Second, being less space-constrained than with more expensive DRAM, PMIO can buffer data across multiple collective I/O calls before writing them back to parallel file systems to further improve I/O performance. Third, we design a two-level log merging approach to reduce communication overhead for data shuffling among MPI processes on compute nodes. Our experimental results with representative MPI-IO benchmarks show that PMIO improves the I/O throughput by up to 121X and 151X for writes and reads respectively on the Perlmutter supercomputer.