MAAF: A Mutual Alignment and Adaptive Fusion Framework for Sketch-Based Image Retrieval

Sketch-based image retrieval (SBIR) remains a challenging cross-modal task due to the substantial modality gap between free-hand sketches and natural photos, as well as large intra-class variations within each domain. To address these issues, we propose Mutual Alignment and Adaptive Fusion framework (MAAF), a novel method that jointly enhances cross-modal representation alignment and adaptive retrieval matching. MAAF comprises two key components: Mutual Knowledge Alignment with Symmetry (MKAS) and Class-Adaptive Distance Fusion (CADF). The MKAS module facilitates bidirectional knowledge exchange between teacher and student networks, enabling the learning of modality-invariant and discriminative embeddings. Meanwhile, the CADF module adaptively integrates Product Quantization (PQ) and Euclidean distances under the guidance of a class-level compactness measure. This adaptive mechanism assigns greater weight to PQ for compact categories to boost efficiency, and higher weight to Euclidean distance for scattered ones to ensure accuracy, balancing retrieval precision and scalability. Extensive experiments on standard ZS-SBIR benchmarks, including Sketchy and TU-Berlin, demonstrate that MAAF consistently outperforms existing methods. It achieves state-of-the-art retrieval accuracy and enhanced generalization to unseen categories, addressing cross modal gaps, the absence of training-test category overlap, and intra-class diversity.