Effective and Efficient Cross-media Knowledge Transfer Through Adapted Invertible Intermediate Space Construction

Cross-media knowledge transfer endeavors to utilize the knowledge or features obtained from the source modality to enhance performance in the target modality. However, the challenge arises from the modality shift that occurs within heterogeneous data. In this paper, we solve the modality shift by introducing a novel adapted invertible intermediate space construction method. Unlike traditional conventional approaches, which typically support limited data domains or tasks, the proposed method explicitly enables knowledge transfer across diverse data domains (e.g., visual images and textual descriptions) and tasks (e.g., single-modal image retrieval and cross-modal image-text retrieval tasks). Particularly, this method extracts multiple features of source and target domains, and then projects them into a shared intermediate space through matrix transformations. Additionally, a discriminative function is utilized to achieve structural alignment. Finally, pairwise similarity is applied in the intermediate space to achieve invertible adaptation across different modalities and tasks. Experiments show that our approach surpasses state-of-the-art methods in both image retrieval and image-text retrieval tasks, validating its competitive effectiveness and efficiency.