Intrepydd: Toward Performance, Productivity, and Portability for Massive Heterogeneous Parallelism
摘要
This paper introduces our ongoing work on the automatic ahead-of-time (AOT) parallelization of Python programs on recent and future hardware systems with massive parallelism and heterogeneity. Our approach is driven by the combination of ML-based type prediction and multi-versioned code generation that guarantees the correctness of our type-specific code optimizations in all cases. While Python is a dynamically-typed language, recent research demonstrated it is highly possible to predict what data types are likely to occur at runtime, by ML-based static prediction and/or runtime type profiling in numerical computation kernels. Given code fragments with predicted data type information, our optimization engine performs automatic parallelization and sophisticated high-level code optimizations for the target system, such as shared/distributed heterogeneous hardware platforms. Our approach introduces novel extensions to the polyhedral compilation to integrate loop and data layout transformations as well as automated selection of CPU vs. GPU code variants. Our preliminary empirical evaluation shows significant performance improvements relative to sequential Python in both single-node and multi-node experiments.