Cost-Performance Analysis: A Comparative Study of CPU-Based Serverless and GPU-Based Training Architectures

This paper presents a comparative evaluation of four serverless training frameworks: SPIRT, MLLess, LambdaML AllReduce, and ScatterReduce, alongside a GPU-based baseline, using CNN models on CIFAR-10. We assess each architecture across training time, cost, communication overhead, and accuracy under consistent experimental conditions. While GPU-based training achieves the fastest convergence and highest accuracy, serverless frameworks offer cost advantages for lightweight models. Optimizations such as gradient accumulation and in-database computation improve serverless performance. Our findings reveal key trade-offs and highlight the potential of GPU-backed serverless platforms for scalable distributed training.