ExBoost: Out-of-Box Co-optimization of Machine Learning and Join Queries

Many real-world machine learning (ML) inference workflows involve data integration steps that join multiple data silos to assemble feature vectors. Such join operations are often expensive and easily become bottlenecks in the end-to-end inference process. We argue that factorizing ML computations and pushing down factorized computations through join can effectively accelerate the end-to-end processing of such queries. However, existing work on factorized ML focuses on the learning processes of traditional ML models and the computation of separate linear algebra operators, with two drawbacks. First, they never considered and evaluated the factorization of the end-to-end deep neural network inference process. Second, existing optimizers did not consider the relative overheads of the decomposable part and non-decomposable part and the alignment of the cost estimation for the SQL processing and ML processing. To close the gaps, we (1) demonstrate an out-of-box AI/ML-SQL co-optimization approach for end-to-end inference workflows where the users specify a SQL query, preprocessing operators, and a pre-trained model exported in ONNX format, and the end-to-end processing will be automatically optimized and synchronized in our ExBoost system; and (2) propose a contrastive learning approach to automatically estimate the potential speedup by applying the factorization strategy.