RNS for DNN Architectures

Modern Deep Learning models keep growing in depth and number of parameters and require a huge amount of elementary arithmetic operations, the majority of which are multiply-add operations. These operations can be very efficiently implemented in the Residue Numbering System: RNS encoding allows for carry-free computations among the different residue channels, with inherent parallelism at the digit processing level. Arithmetic circuits for addition and multiplication become smaller and can operate on higher frequencies and with lower power consumption. In this Chapter, the basic RNS arithmetic operations and their hardware implementation are described. Moreover, RNS-based DNN architectures reported in the literature are presented and compared.