3D Human Target Tracking and Localization Based on Millimeter Wave Radar and Visual Fusion

In recent years, 3D object detection has become an important component in the fields of autonomous driving and mobile robots. Current vision-based methods may fail to achieving reliable positioning of a 3D object due to the sensing limitations of a camera sensor. In this paper, we consider a more practical object perception pipeline using combined information from both a monocular camera and a 4D millimeter-wave radar (MMWR). Initially, the human body is detected in the 2D image plane of camera to aid the human body segmentation in 4D radar point clouds. Then the detection results from both the camera and the MMWR, including the doppler velocity of the point cloud, are used as measurement to feed an Kalman filter estimator backend to realize the rapid localization of human targets.