Maritime Vessel Detection and Classification in Harbor Environment Using Deep Learning

Automatic surveillance of docks, harbors, and seaports is considered crucial for ensuring smooth port operations. The identification and tracking of maritime vessels in these environments are generally facilitated by Radar and Automatic Identification System (AIS). However, radar faces challenges in detection of small non-metallic vessels, as well as there are other issues such as radar signal clutter, high electromagnetic radiation, and higher costs. Similarly, AIS also face challenges such as device malfunctioning or illegal manipulation etc. Consequently, cameras have been increasingly used in maritime traffic management systems. In recent years, Convolutional Neural Networks (CNNs) have shown significant progress in object detection and classification tasks. Though, obtaining large scale maritime datasets with focus on harbor environment still remains a challenge. In this study, experimentation with various pre-trained state-of-the-art CNN models were conducted on images captured from harbor environments in Karachi and Grand Canal in Venice. The dataset presented several challenges such as high-density traffic, occlusions, shadows, background infrastructure, wave motion, boat wakes and reflection on water surface. The pre-trained CNNs were finetuned using a combination of small datasets to obtain both detector and classifier which outperformed previously published results on the MarDCT dataset. Specifically, the proposed model achieved a DR (Detection Rate) of 0.81, FAR (False Alarm Rate) of 0.07 and an average accuracy of 98.78% on the MarDCT dataset.