Cross Attention Graph Matching Network for Image-Text Retrieval

Image-text retrieval is a basic cross-modal task whose main idea is to learn image-text matching. As graph convolutional networks are widely used in visual semantic tasks, graph structures are used to represent unstructured information such as information on nodes and correlation between nodes. In this paper, we propose an image-text retrieval model (CAGMN) based on cross attention graph matching. We use the significant regions in the image and the words in the text to model the graph nodes. Then, we use the graph convolutional network respectively to deduce the relationships within the modes and extract the relationships within the modes. At the same time, cross-attention feature extraction method is introduced to promote cross-modal flow of matching information between image regions and words, extract features containing cross-modal matching information, and make full use of intra-modal and inter-modal information. Finally, the graph structure matching and image-text global similarity calculation are carried out. At the same time, the graph structure matching information and global similarity information are used to learn the image-text matching relationship at different levels.