Hardware/Software Co-Design for Traffic Sign Inference on Edge Using PYNQ

Traffic sign classification is the real-time safety-critical task in driver assistance systems and self-driving vehicles which is realized via the deep learning approach. Edge-based solution is a feasible one for implementing such critical tasks which offers low latency and privacy in inference process by utilizing hardware acceleration. LeNet-5 was one of the exemplar architectures of Convolutional Neural Network (CNN) used for traffic sign classification and realizable for edge platforms. This paper proposes an efficient Overlay for LeNet-5- based traffic sign classification utilizing the PYNQ framework, based on hardware/software co-design. Utilizing Vivado HLS, LeNet-5’s computationally intensive tasks are incorporated in the custom IP core, and VivadoHLx is used to create the Overlay. The pretrained LeNet-5 model is integrated with the Xilinx ZYNQ 7020 System-on-Chip (SoC) on the heterogeneous PYNQ-Z2 evaluation platform using the German Traffic Sign Recognition Benchmark (GTSRB). This evaluation assesses the Overlay’s ability to make inferences using Python. According to the outcomes of our hardware/software co-design method, classifying 111 images per second while operating at a conservative 100 MHz speed is around 314x faster than a 650 MHz software implementation.