Text semantic-guided adaptive feature aggregation for image-text retrieval
摘要
The image-text retrieval task aims to build a semantic bridge between images and text. The primary challenge lies in accurately measuring image-text similarity. Existing methods depend on pre-trained object detectors to extract image region features and Transformers for text processing. This approach often results in inconsistent semantic distribution during cross-modal alignment. In addition, there are significant differences in the representation forms and distribution of images and text. In order to accurately represent the real world, images contain a large amount of detail and background information, resulting in higher information redundancy than text. This redundant information not only increases the computational burden but also tends to cause inaccurate alignment issues. To address these issues, this paper proposes a Text Semantic-Guided Adaptive Feature Aggregation (TGAA) framework based on Transformer architecture. The framework designs a text semantic-guided feature purification module, which leverages both text supervision and image self-supervision to guide the filtering of redundant image features, thereby enhancing the compactness and discriminability of image representations. An adaptive feature aggregation strategy is then employed to further enhance the expressiveness of global features without increasing architectural complexity. The model conducted extensive experiments on two benchmark datasets, Flickr30k and MS COCO. It achieved a maximum improvement of 29.8% in the RSUM compared to the baseline. These results highlight the effectiveness of the proposed framework and its potential in cross-modal retrieval tasks, confirming its competitive retrieval performance.