Portable Containerized MPI Application Using UCX Replacement Method

In high-performance computing (HPC), using Linux containers can improve the convenience of system management and utilization, as well as facilitate the adoption of HPC DevOps and Clouds. However, utilizing containers with MPI applications poses some challenges in building portable MPI containers that can leverage various high-performance interconnect hardware available on different systems. A common solution is to replace the entire MPI stack in the container, but this may introduce compatibility issues between the host and container libraries. An alternative and more flexible solution is to inject host-shared libraries into the container and use them in conjunction with the container’s MPI stack, commonly referred to as a hybrid approach. In this paper, we propose a hybrid solution that uses the UCX communication framework as an intermediary between MPI and various high-end interconnects. This solution simplifies the container-building process while still achieving near-native communication performance. The flexibility of the proposed method was tested by performing a cross-version injection experiment. The results showed a good degree of forward/backward compatibility between versions, which can also refer to good portability as well. We evaluated the performance of our solution using communication micro-benchmarks and application benchmarks, and found that it achieved near-native communication performance. We also compared our solution with a similar solution using the Libfabric communication library, and confirmed that they have similar pros and cons with similar performance characteristics in most cases.