Approximate Communication in Network-on-Chips for Training and Inference of Image Classification Models

Approximation is an emerging design methodology for reducing power consumption and latency of on-chip communication in many computing applications. However, for training and inference of image classification models, existing approximate communication techniques achieve modest improvements in these metrics. In this chapter, we propose an approximate communication technique to improve the efficiency of on-chip communications by exploring the error tolerance of the model during training and inference, resulting in better overall performance for the operations. This is achieved by taking advantage of three approximation opportunities, which are quantization, contrast reduction, and matrix sparsification. The proposed approximate communication technique is implemented with quality control and data approximation techniques for quantization and contrast reduction during inference. The implementation also includes a dual-matrix compression method to further reduce the size of sparse matrices in on-chip communication. The combined effect of data approximation and compression reduces the number of flits in each data packet as well as the on-chip communication while maintaining excellent image classification accuracy for both training and inference. The detailed evaluation shows that compared to the state-of-the-art approximate communication technique, the proposed technique achieves 35% and 31% reduction in dynamic power consumption and network latency, respectively, for training. In terms of inference, the proposed technique reduces dynamic power consumption and network latency by 27% and 26%, respectively, with less than 0.99% accuracy loss.