GPU cluster dynamics: insights from Alibaba’s 2023 trace release

In this paper, we present a comprehensive analysis of GPU cluster traces from Alibaba, released in 2023, focusing on understanding the detailed settings of nodes and pods and the important numbers related to them. By examining the configurations of 1,523 nodes, predominantly GPU-based, we identify a wide array of configurations and GPU models, highlighting a balanced interplay between CPU cores and RAM across all nodes while also noting a decoupling between CPU/RAM and GPU-specific metrics in GPU nodes, indicating a nuanced approach to workload placement. Our investigation extends to an analysis of 8,152 pods, revealing diverse configurations and a significant prevalence of latency-sensitive pods, indicative of a cluster geared towards timely resource provisioning. The presence of Failed and Pending pods, however, underscores the complexity and challenges inherent in scheduling efficiency and resource allocation. By delving into both node and pod configurations and metrics, this study sheds light on the operational characteristics and workload patterns within the cluster. The insights gleaned from this analysis are beneficial for researchers, system designers, and operators, providing a nuanced understanding of the cluster’s workings. This, in turn, lays the groundwork for future research and optimization efforts, ultimately contributing to the advancement of GPU cluster management and operation.