A Robust 3D Object Tracking Method Based on Fusion of Millimeter-Wave Radar and Camera

For the task of 3D multi-object tracking using millimeter-wave (MMW) radar and camera fusion, inherent limitations such as the camera's restricted field of view and the radar's low resolution are exacerbated by environmental factors like low light, dense fog, and noise interference. These conditions often lead to prolonged detection loss and significant detection errors, posing substantial challenges to tracking accuracy and robustness.To address this issue, this paper presents an in-depth study on robust 3D multi-object tracking technology for MMW radar and camera fusion. While existing research has improved performance through track-to-track fusion, these methods exhibit an over-reliance on uninterrupted sensors during trajectory interruptions and fail to fully leverage mutual information between sensors to eliminate data redundancy. This method integrates two core strategies: First, a robust tracking strategy employing short-term state extrapolation based on historical information, which dynamically adjusts fusion weights during sensor trajectory interruptions to ensure continuous and stable tracking despite data gaps. Second, a correlation-based track-to-track fusion strategy that quantifies inter-sensor dependencies by calculating cross-covariance matrices, thereby reducing redundancy and enhancing accuracy during the fusion process. Experimental results under both normal conditions and various extreme scenarios demonstrate that the proposed AWA-MOT method outperforms existing approaches in both tracking accuracy and robustness.