Lidar-camera range-view fusion for 3D object detection in autonomous driving
摘要
In recent years, LiDAR-Camera fusion for 3D object detection has emerged as a research hotspot, owing to its superior performance compared to single-sensor approaches. However, the vast majority of methods adopt fusion in the point, voxel, or BEV domains, which may lead to domain discrepancies and misalignment, resulting in suboptimal detection accuracy. In contrast, this paper introduces a novel range-view fusion strategy, leveraging its unique advantages—data homogeneity, accurate feature alignment, and high efficiency—over other methods. We introduce the Range-view Pyramidal Fusion network (RPfusion) as a plug-and-play framework compatible with various LiDAR-based detectors. It incorporates two dedicated modules—Appearance Feature Fusion (AFF) and Range-Guided Cross-Layer Modulation (RGCLM)—to effectively tackle the inherent challenges of range-view fusion: substantial object scale variations and feature vanishment caused by occlusion. First, the AFF module adaptively integrates complementary geometric and information across different scales using dilated cross-attention. Next, RGCLM module is proposed to extract rich contextual information for comprehensive perception and enhance features of occluded targets. The output features can globally decorate raw point clouds to boost detection accuracy of any LiDAR-based model. Built upon RPFusion, we further develop a novel two-stage detector—RPfusion-Det that additionally incorporates an RoIfusion module for fine-grained local feature aggregation. Extensive experiments on the KITTI and NuScenes datasets demonstrate that RPFusion can consistently enhances various detection models, while RPfusion-Det achieves superior accuracy with efficient inference.