Transparent Remote OpenMP Offloading Based on MPI

In this work, we present an efficient mechanism which allows unmodified OpenMP applications to leverage the computational resources of any node in a cluster through the OpenMP device interface. Remote CPUs and remote accelerators such as GPUs are all available to offload code portions to, completely transparently, as if they belonged to the host node that executes the application. This is possible by virtualizing remote resources to appear as local devices and introducing a device-agnostic communication mechanism that forwards data requests to/from the remote nodes. The communication layer is based on MPI, eliminating the need for custom request servers, while providing excellent portability and optimized performance for a variety of network fabrics. Moreover, our implementation allows concurrent offloading to any combination of local and remote devices.