Masked frequency-color fusion network for video instance-level hazy lane detection

Existing video instance-level lane detection (ILD) methods often assume that the input videos are clear. However, videos captured in hazy weather conditions are inevitably corrupted by the haze, thereby degrading the video instance-level lane detection accuracy. In this work, we address the hazy video ILD task by fusing masked frequency-level phase information and color information. Bearing the assumption that haze-free features are more helpful for ILD, we devise a multi-task network, which contains a frequency-level phase branch, a color-dehaze branch and a color-lane branch. We then concatenate phase features with color-dehaze feature map, and disentangle the resulted features into two components: one is passed to assist the prediction of instance-level lane detection, while another component is used for predicting haze-free counterpart for the input key frame. After that, a masked self-attention module is proposed to effectively integrate the disentangled features, the phase features, and the color-lane features, followed by the final instance-level lane detection. In the experiments, we build an annotated dataset VIL-hazing67 for ILD from hazy videos. The comparison with state-of-the-art video ILD methods as well as image ILD methods clearly show that our network achieves superior performance in terms of instance-level lane detection from hazy videos. Our source code is available at: https://doi.org/10.5281/zenodo.12524860.