Joint Similarity Guidance Hash Coding Based on Adaptive Weight Mixing Strategy For Cross-Modal Retrieval
摘要
There is a continuous and explosive growth of multimodal data. Efficient cross-modal hashing retrieval is of significant importance in conserving computational resources.To further enhance the attention to informative data within modalities and capture the semantic correlations in cross-modal data, we propose an enhanced deep Joint-Semantics Reconstructing Hashing algorithm, which is the Joint Similarity Guidance Hash Coding Based on Adaptive Weight Mixing Strategy(JSGHCA). The algorithm focuses on delving deeper into the correlations of the data in cross-modal. We introduce the adaptive weight mixing strategy to construct the semantic affinity matrix, so that the matrix can identify each modal data with specific weight in each batch. At the same time, in the process of the hash code generation, we introduce collaborative attention mechanism. It helps the model to pay more attention to the local information of each modality, thereby capturing the semantic features within each modality more accurately. Additionally, it enables the model to jointly process the attention across different modalities and extract shared semantic features more precisely. Experimental results show that the proposed model is significantly better than the deep joint semantic reconstruction hash algorithm on multiple benchmark datasets.