Multi-task Collaborative Network for Image-Text Retrieval

Image-text retrieval aims to capture semantic relevance between images and texts. Most existing approaches rely solely on the image-text pairs to learn visual-semantic representation through fine-grained alignments while neglecting the potential beneficial impact of unimodal tasks on cross-modal retrieval. To this end, we present a Multi-Task Collaborative Network (MTCN) that leverages the synergy between multiple tasks to enhance the performance of image-text retrieval. Specifically, we introduce three unimodal tasks, including text-text matching, image multi-label classification, and text multi-label classification, and train together with target task image-text retrieval from the perspective of semantic constraints. Additionally, we employ a modality interaction module for image-text retrieval to discover interrelationships between these two modalities. Subsequently, a cascaded graph convolutional network combined with a multi-layer perceptron is used to infer the correlation scores between images and texts. We conduct comprehensive experiments on two benchmark datasets, Flickr30K and MSCOCO, and the quantitative and qualitative experimental results demonstrate the effectiveness of the proposed method. The source code is available at https://github.com/FlyCuteBird/MTCN .