Reusing Your Prepared Data: An Informed Cache for Accelerating DNN Model Training

In deep learning training, CPU-intensive data preprocessing often leads to CPU bottlenecks, and expensive GPUs cannot be fully utilized, thus degrading end-to-end training performance. In general, CPUs are used to doing preprocessing and GPUs are used to training the model. We propose a new caching algorithm for AI model training, named HCache, which uses the DLT-Informed caching approach to improve the reuse of cached data and the usage of memory. DLT-Informed Caching approach can make intelligent caching decisions by identifying data discrepancies in different preprocessing stages. Our evaluation shows that HCache can achieves about 2 \(\times \) speedup compared with the state-of-the-art CoorDL and Quiver in the training of computer vision models while maintaining comparable accuracy.