On-road obstacle detection in real time environment using an ensemble deep learning model

Obstacle detection on road is a challenging task in autonomous vehicle driving. Although obstacle detection is carried out with the help of sensors which are accurate and precise in real-time, they are not cost effective and computationally intensive. So, a computer vision and deep learning-based approach can be considered as a potential alternative. The proposed system is an ensemble of two instance segmentation algorithms namely You Only Look Once v7 (YOLOv7) and Mask Region-Based Convolutional Neural Network (Mask R-CNN) used to detect various obstacles on road. The system was tested on popular obstacle detection datasets such as INRIA, KAIST, and COCO2017. The ensemble model achieved mean average precision score of 90.1, 95.4 and 88.9 and mean intersection over union score of 93.3, 89.7 and 91.4 for KAIST, INRIA and COCO datasets respectively. A custom dataset for detecting obstacles was developed and the proposed model was tested on the custom dataset as well.