SmartDriveNet: an integrated approach for robust driving perception in unstructured environments and adverse weather conditions
摘要
Accurate and robust environmental perception is essential for the safe and efficient operation of autonomous vehicles and Advanced Driver Assistance Systems (ADAS). However, unstructured road environments and adverse weather conditions, such as rain, fog, snow, potholes, cracks, and chaotic traffic, pose significant challenges to current detection systems. To address these issues, we introduce SmartDriveNet, a novel multitasking panoptic driving perception system designed to excel in such complex scenarios. SmartDriveNet performs seven critical perception tasks vital for autonomous driving and ADAS, including traffic light recognition, traffic sign recognition, pothole and crack detection, drivable area segmentation, lane detection, and traffic object recognition involving rider, person, animals, and different types of vehicles. Key contributions of the work include the integration of Generalized ELAN (GELAN), enhanced with programmable gradient information (PGI) to extract a wider range of features, and the Density Adaptive Attention Mechanism (DAAM), which dynamically adapts to diverse contexts, improving feature refinement and robustness across all tasks. A carefully designed loss function and training strategy ensure balanced multitasking performance in real-world conditions. To train and evaluate SmartDriveNet, we present the Indian Traffic and Adverse Weather Dataset (ITAWD), comprising 24,236 images across 47 classes that capture diverse road and traffic scenarios, including adverse weather and challenging conditions. SmartDriveNet achieves state-of-the-art performance in both accuracy and speed on the challenging BDD100K dataset and the custom ITAWD dataset, demonstrating its practicality and reliability for real-time deployment, contributing to safer and more robust transportation systems. ITAWD can be accessed at https://universe.roboflow.com/shilpa-mocpz/itawd.