An improved object detection network based on LIDAR point cloud and camera

The range of applications for Light Detection and Ranging (LiDAR) has been expanding, especially in object detection. But they usually depend on one modality and cannot extract information from others. LiDAR and camera multimodal fusion combines data from two sources, greatly increasing detection precision. This paper proposes a new network called the Convergent Attention-Enhanced Camera-LiDAR Object Candidates System (CAECs). It is a decision-level fusion architecture for target detection. First, the CAECs network uses an advanced candidate encoding mechanism. This mechanism sifts through and saves prime candidates from both 2D and 3D detectors. It forms a comprehensive feature tensor and avoids missing crucial detections. Second, we use AgileSightNet to improve feature relevance and strengthen important data. AgileSightNet includes layered channel fusion and an attention scheme. The tests on the KITTI benchmark show that our method performs better at detecting pedestrians and cyclists. It improves accuracy by 6.43% and 6.26% compared to the existing 3D multimodal networks. Compared to single-modal 3D networks, our method improves detection accuracy by 13.03% and 5.97%. This shows better precision and robustness in LiDAR point cloud applications.