Hierarchical-Caching-Driven Distributed Architecture for Accelerating Model Training
摘要
Scaling deep learning to industrial dimensions has made distributed training indispensable. However, its efficiency is often hindered by the long-tail distribution of parameter access, caused by extreme skew. Conventional approaches mainly rely on worker-side caches, but these static and rigid designs fail to adapt to dynamic traffic patterns, leading to persistent network congestion and underutilized GPUs. To address this challenge, we propose M-Cache, a switch-assisted, multi-tier caching framework that operates hierarchically and adapts in real time. By embedding a lightweight programmable cache directly in the datapath and coordinating it with adaptive worker buffers, M-Cache ensures that frequently accessed parameters are placed at the optimal layer where they are most needed. Experimental results demonstrate clear benefits: compared with state-of-the-art static methods, M-Cache reduces parameter pull traffic by up to 70% and accelerates end-to-end training by as much as 1.92 \(\times \) , showing that incorporating caching into the network fabric effectively mitigates skew and enhances training efficiency.