A Comprehensive Benchmark for Evaluation Inference Strategies on Large Scale Tabular Datasets
摘要
This study examines three inference strategies—serial, multi-core parallel (Joblib), and distributed memory (MPI)—to address scalability challenges in applying machine learning models to large tabular datasets. Using the HIGGS dataset as a representative benchmark of high-volume scientific data, a Random Forest classifier was trained and validated with performance metrics of 73.3% accuracy and 81.25% ROC-AUC. The results offer concrete guidance on selecting appropriate inference backends based on system architecture and dataset size, particularly for practitioners in high-performance computing and scientific machine learning domains.