ESIMD GPU Implementations of Deep Learning Sparse Matrix Kernels

We demonstrate that explicit SIMD programming on GPUs can outperform traditional programming environments such as CUDA and SYCL for three sparse matrix computations found in deep learning applications. Intel oneAPI’s Explicit SIMD (ESIMD) SYCL extension API allows for simpler vectorization of arithmetic and memory operations which is critical in achieving good performance. We explore sparse matrix operations relevant to deep learning applications, namely the sparse-dense matrix multiplication (SPMM), the sampled dense-dense matrix multiplication (SDDMM), and the composition of the SDDMM with SPMM (FusedMM). Our ESIMD optimizations target the Intel Data Center GPU Max 1550. We evaluated performance on the test data set used by previous work, and our implementation outperforms state-of-the-art CUDA implementations on the latest NVIDIA hardware by up to a factor of 6.14. Additionally, our proposed implementation outperforms Intel’s oneMKL implementation on Intel’s GPU.