LiDAR 3D Object Detection in FPGA with Low Bitwidth Quantization

Detection of objects from LiDAR point clouds is an important task for several applications, like autonomous driving. Deep neural network models for LiDAR processing require a large processing computing capacity and storage. So, real-time execution of these models requires a high-performance computing platform on-board. To reduce the stress over the onboard computer, some proposals consider lite models at the cost of some accuracy. Instead of model reengineering, an integrated model/architecture optimization can reduce the computing requirements without compromising accuracy. This work optimizes a 3D object detection CNN model (PointPillars) and proposes an FPGA-based accelerator for efficient model inference. Fixed point quantization was performed for both weights and activations, with a vast exploration of the quantization possibilities for the selected model. This allowed us to improve the hardware accelerator while maintaining good detection accuracy. The proposed accelerator was implemented with a hybrid quantization with 8 and 2 bits for weights and 8 bits for activations. The final architecture achieved a throughput of 15.6 frames per second using 85K LUTS, 149 DSPs and 200 BRAM.