Robust multimodal 3D object detection: overcoming weather challenges in autonomous driving perception systems
摘要
Multimodal 3D object detection, fusing LiDAR and camera data, is crucial for autonomous driving. However, existing methods suffer significant performance degradation in adverse weather due to feature mismatches caused by compromised sensor data. For instance, LiDAR is obscured by snow, while cameras struggle in low light. State-of-the-art approaches, while accurate in ideal conditions, lack robustness for real-world applications. To address this, we introduce a comprehensive benchmark to evaluate model robustness across four adverse weather conditions. We then propose a synergistic framework combining robust fusion design and data augmentation to mitigate asymmetric sensor degradation. Extensive experiments show our framework effectively handles imbalanced feature degradation, significantly improving system robustness. Our findings provide critical insights for optimizing multimodal perception in autonomous driving under adverse weather conditions, paving a more reliable pathway for real-world deployment.