RCBEVD: Radar-Camera Fusion in Bird’s Eye View for Detection with Velocity Estimation

Radar and camera sensors are commonly used in the perception stack of autonomous driving systems due to their maturity, cost-effectiveness, and robustness. The outputs from these sensors are often fused to generate more accurate perception results, due to their complementary features. In this paper, we propose a method, RCBEVD: Radar-camera fusion in Bird’s Eye View (BEV) representation for detection with velocity estimation. We introduce a novel clustering method for radar point clouds alongside a tracking algorithm to estimate the true radar point’s velocity. The estimated true radar velocities are fused with 2D object detection bounding boxes to give BEV detections. RCBEVD is evaluated on the nuScenes dataset benchmark using the Average Velocity Error (AVE) and Average Precision (AP) metrics. We outperform the baseline method CenterFusion’s Mean Average Velocity Error (mAVE) \(0.696\,\text {m/s} \,(-0.690\,\text {m/s})\) , an improvement of \(49.8\%\) on the nuScenes validation dataset.