SodStereo: An Effective Multi-task Learning Network for Stereo Matching and Salient Object Detection

Disparity estimation has a wide range of applications in vision tasks and robotics. Due to the development of deep neural networks, disparity estimation models have been implemented end-to-end and achieve remarkable performance. To achieve better results on the regions of non-textures, boundaries, and tiny details, it is necessary to effectively combine global context information. However, current models rely on intricate cascade structures or stacked 3D convolutional layers that demand substantial computational resources. To address these issues, we present a multi-task network called SodStereo, consisting of a backbone disparity network and a Salient Object Detection network. In this paper, we aim to integrate Salient Object cues from images into the disparity estimation task. To accomplish this, we utilize the feature maps generated by the SOD network as attention weights. These weights serve to enhance pertinent information and mitigate redundant data within the cost volume. This approach eliminates the necessity for stacking 3D convolutions and intricate cascade structures prior to disparity regression. Additionally, our validation on the KITTI2012 and KITTI2015 datasets demonstrates the network’s relative effectiveness.