Adaptive Graph Attention Hashing for Unsupervised Cross-Modal Retrieval via Multimodal Transformers

Unsupervised cross-modal hashing retrieval has been extensively studied due to its advantages in storage, retrieval efficiency, and label independence. However, there are still two obstacles to existing unsupervised methods: (1) Existing unsupervised methods suffer from inaccurate similarity as simple features do not describe fine-grained multimodal relationships. (2) Existing methods suffer from unbalanced multimodal learning due to the different coding capabilities of different modal networks. To address these obstacles, we devised an effective Adaptive Graph Attention Hashing (AGAH) for unsupervised cross-modal retrieval. Firstly, we use the multimodal transformer model CLIP to extract cross-modal fine-grained features and exploit multiple data similarities to mine similar information from different perspectives in multi-modal data and perform similarity enhancement. In addition, we present an adaptive graph attention hashing module to assist in generating hash codes, which uses an attention mechanism to learn relation-based similarity from image-text modality. It aggregates the essential neighborhood message of neighboring data nodes through the graph neural networks to generate more discriminative hash codes. Sufficient experiments on three benchmark datasets demonstrate that the proposed AGAH outperforms existing advanced unsupervised cross-modal hashing methods.