In recent years, with the rapid development of autonomous driving and intelligent transportation, 3D object detection technology has gradually become a research hotspot. However, when the target objects are small or obstructed, the point cloud data obtained by radar is often insufficient, resulting in incomplete structural and semantic information, leading to missed detections and false detections. To address this challenge, this paper proposes a multimodal 3D object detection method based on point density fusion (Point Density Fusion for Multimodal 3D Object Detection, PDFusion). The algorithm introduces a Point Density Information Fusion Module (PDIF) and a Adaptive Gating Circuit Fusion Module (AGCF), utilizing point cloud density features and an adaptive gating fusion module to effectively leverage the geometric features of point clouds and the semantic information from camera images. This approach improves detection accuracy for small and obstructed objects. The AGCF module uses a self-attention mechanism to achieve efficient fusion of multimodal features, while the PDIF module further optimizes candidate boxes based on point cloud density information. Experimental results show that the proposed model achieves excellent performance on the KITTI validation set, particularly in Hard samples, where PDFusion reaches 3D detection accuracies of 85.95%, 60.11%, and 71.44% for the car, pedestrian, and bicycle categories, respectively. Additionally, the model demonstrates significant advantages on the Easy and Moderate samples of the KITTI test and validation sets, confirming its effectiveness and generalization in enhancing multimodal 3D object detection accuracy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Point Density Fusion for Multimodal 3D Object Detection

  • Ziyang Peng,
  • Jianxu Mao,
  • Wei He,
  • Caiping Liu,
  • Zhenyu He,
  • Ziming Tao,
  • Yaonan Wang

摘要

In recent years, with the rapid development of autonomous driving and intelligent transportation, 3D object detection technology has gradually become a research hotspot. However, when the target objects are small or obstructed, the point cloud data obtained by radar is often insufficient, resulting in incomplete structural and semantic information, leading to missed detections and false detections. To address this challenge, this paper proposes a multimodal 3D object detection method based on point density fusion (Point Density Fusion for Multimodal 3D Object Detection, PDFusion). The algorithm introduces a Point Density Information Fusion Module (PDIF) and a Adaptive Gating Circuit Fusion Module (AGCF), utilizing point cloud density features and an adaptive gating fusion module to effectively leverage the geometric features of point clouds and the semantic information from camera images. This approach improves detection accuracy for small and obstructed objects. The AGCF module uses a self-attention mechanism to achieve efficient fusion of multimodal features, while the PDIF module further optimizes candidate boxes based on point cloud density information. Experimental results show that the proposed model achieves excellent performance on the KITTI validation set, particularly in Hard samples, where PDFusion reaches 3D detection accuracies of 85.95%, 60.11%, and 71.44% for the car, pedestrian, and bicycle categories, respectively. Additionally, the model demonstrates significant advantages on the Easy and Moderate samples of the KITTI test and validation sets, confirming its effectiveness and generalization in enhancing multimodal 3D object detection accuracy.