AI Runtime: Elastic Compute for AI Workloads

When it comes to graphics processing units (GPUs), people undoubtedly think of Copilot or ChatGPT. No doubt that large language models are powered by a massive number of GPUs, and we will also discuss AI models in this chapter. One lesser-known limitation is the limited support for training traditional machine learning models on serverless CPU compute. The reason is that serverless is using the Spark Connect architecture. As shown in Figure 10-1, the Spark Connect architecture decouples the Spark driver and the application layer or the execution environment. While the client-server architecture allows the core Spark engine and the execution environment to continue to evolve independently, the drawback is that the application must now access Spark via API, and it no longer has access to the low-level RDD API.