A Unified and Energy-Efficient Depthwise Separable Convolution Accelerator

Lightweight convolutional neural networks (CNNs) reduce computational workloads, making them suitable for embedded devices with limited hardware resources compared to conventional CNNs. Depthwise separable convolution (DSC) serves as the fundamental convolution unit of lightweight CNNs. This paper introduces a hardware accelerator tailored for DSC in Application-Specific Integrated Circuit (ASIC), featuring a unified engine supporting both depthwise convolution (DWC) and pointwise convolution (PWC) with high hardware utilization. It ensures 100% processing element (PE) array utilization for DWC and achieves up to 98% utilization for PWC while minimizing latency. By partitioning the input feature map (ifmap) Static Random-Access Memory (SRAM) into three banks, memory access is streamlined. Furthermore, a data scheduling strategy, along with a multiplexed registers (MR) bank based First-In-First-Out (FIFO) system between adjacent PEs, is implemented to maximize data reuse and reduce latency. This work is implemented in a 22 nm FDSOI technology and validated on the CIFAR10 dataset using the MobileNetV1 architecture. The proposed DSC accelerator can operate at 1 GHz, exhibiting an energy efficiency of 5.07 (3.96) TOPS/W and an area efficiency of 519.2 (461.52) GOPS/mm \(^{2}\) for DWC (PWC) at 0.8 V. Scaling the supply voltage down to 0.5 V increases the energy efficiency to 13.64 TOPS/W for DWC and 10.64 TOPS/W for PWC.