Adaptive omni-supervised learning for robust object detection and tracking in autonomous driving
摘要
Autonomous driving systems require precise object detection, segmentation, and tracking to ensure safe navigation. However, these tasks typically depend on expensive labeled datasets and experience challenges such as occlusions, blind spots, and dynamic traffic conditions. To mitigate these limitations, the proposed research introduces an adaptive omni-supervised learning framework designed to reduce dependency on manual annotations by automatically generating labels from both labeled and unlabeled data sources. Our approach performs pixel level labeling by integrating Recognize Anything Model (RAM), Grounding DINO (GDINO) and Segment Anything Model (SAM) for 2D image segmentation and Complex Yolo for 3D images obtained from LiDAR data. The framework also incorporates a multi-modal fusion network that integrates 2D visual information with 3D LiDAR-based spatial representations. The fusion output enables a richer and more accurate understanding of the surrounding environment. Further, this research incorporates Kalman filter based object tracking, where process and measurement noise parameters are empirically optimized to maintain robustness under dynamic and noisy driving conditions. Experiments on real-world driving datasets demonstrate 82% precision, 0.86 Intersection over Union (IoU) and 85% Multiple Object Tracking Accuracy (MOTA). In comparison with standalone 2D and 3D methods, the proposed system demonstrates improved robustness under challenging driving conditions, thereby offering a more reliable and scalable solution for real-world autonomous navigation.