Research on Monitoring Technology Based on the Fusion of 4D Millimeter Wave Radar and Machine Vision

Advanced monitoring system requires the sensing module to cover the whole target and the whole working condition, but the traditional monocular camera-based surveillance system can hardly meet this requirement. To address the limitations of a single sensor, a target detection method is designed based on 4D millimeter wave radar and monocular camera fusion. Specifically, we use the DBSCAN algorithm and the Yolov5 target detection algorithm to detect valid targets in radar data and camera data, respectively. Then we construct the coordinate system transformation model and fit the homography matrix using the homography transformation principle and the least-squares estimation method. Finally, a weighted distance-based global nearest neighbor data association algorithm is designed and compared with its method. The detection precision of the proposed association algorithm reaches 81.0%, which is higher than other traditional methods. The experimental results indicate that the detection method is adequate for practical monitoring applications.