A CNN Accelerator on RISC-V-Based SoC for Latency Constrained Edge Networks

Due to the limitation of bandwidth and latency, edge devices are very common platforms for computation of various resource-intensive computations like deep learning techniques used in different domains like computer vision, audio and video processing, etc. Hence various smaller scale deep learning models have already been proposed for execution on edge devices. Either the accuracy or latency of these lightweight models is not sufficient for using in different mission-critical applications. Instead of reducing the resource requirement, we can also update the architecture of the computing platform. System on chip (SoC) can be a good computing platform which is an integration processing system (PS) and field programmable gate array (FPGA) and provides hardware-software co-design. FPGA provides inherent parallelism, on field programmability which not only makes the system flexible but also useful to accelerate various sub-modules of deep learning inference engines. In traditional SoC, PS contains ARM Cortex series processors where the instruction set cannot be customized. Customization of instruction sets in RISC-V-based processor reduces the memory and I/O interaction with the processor and reduces the overall latency. In this work, we have developed an accelerator for deep learning or convolutional neural network (CNN) on RISC-V-based SoC which reduces the overall computation latency compared to the state-of-the-art solutions. The performance of our proposed architecture has been evaluated using the miniVGG CNN model.