Hardware–Software Codesign of an Adder-Tree Type CNN Accelerator

As deep learning applications based on convolutional neural networks (CNNs) are increasingly popular in embedded systems, the demand for a customized CNN accelerator, called neural processing unit (NPU), increases since it promises higher performance per watt than a graphics processing unit (GPU). We introduce a hardware–software codesign methodology of a CNN accelerator and present the design of an adder-type NPU, called MIDAP, as an example. The baseline datapath is designed to maximize the utilization of the MAC units by enabling fully pipelined execution of layers without dynamic resource contention. After deciding the baseline datapath, we define a high-level behavior model based on which a high-level compiler and the system simulator are built concurrently. By modifying the simulation models of the hardware units, we could explore the design space of NPU microarchitecture easily. In addition, we could evaluate the effect of hardware extensions to support various types of non-convolutional operations that recent CNN models use widely such as depth-wise convolution and global pooling. After the final datapath is determined, we perform the remaining design steps for NPU implementation: control structure design, low-level compiler design, and RTL synthesis. The end-to-end performance of Inception V3 is expected to be close to 2.0 TOPs/W.