Design of Accelerators for Combined Infrared and Visible Image Target Detection Based on Deep Learning

The combined infrared and visible target detection based on deep learning attracts much attention because of the satisfactory performance. However, the improved capability of multi-modal detection results in a dramatic increase in the amount of computation and the number of parameters. Overly large and complex model structures are not favorable for deployment on embedded platforms such as Field Programmable Gate Array (FPGA). In this paper, we design a accelerator for joint detection of infrared and visible targets based on the ZYNQ architecture. The overall model size is reduced by quantifying the model parameters and intermediate feature maps. By adding multiple input and output buffer blocks and slicing the input data, the programmable logic (PL) achieves multiple parallel computations. In addition, the convolution layer is combined with a batch normalization layer to improve on-chip resource utilization and computational parallelism. On the AX7Z100 platform, the designed joint infrared and visible image target detection accelerator achieves an energy-efficiency ratio of 4.53 GOP/s/W, with a power consumption of only 5.55 W. The accelerator is 6.25 times more energy efficient than the Intel Core i5-9400F platform and consumes only 8.56 \(\%\) of its power.