DRLHomo: Disentangled Representation Learning for Cross-Modal Homography Estimation
摘要
Cross-modal homography estimation is a fundamental task in computer vision. However, the significant modality gap between multimodal images poses a considerable challenge for this task. In this paper, we propose a novel framework, named Disentangled Representation Learning for Cross-modal Homography Estimation (DRLHomo), to address this challenge. Specifically, we design a feature disentanglement network to explicitly extract modality-shared and modality-unique features of image pairs using shared and unique feature encoders, respectively. Similarity loss and difference loss are employed to supervise this process. Furthermore, to prevent pathological disentanglement, where shared features are dominated by noise and unique features capture all modality information, we incorporate a decoder with reconstruction loss to enhance feature disentanglement. Ultimately, the homography is estimated based on the obtained modality-shared features. Extensive experiments on the GoogleMap and GoogleEarth datasets validate the effectiveness of DRLHomo in mitigating the modality gap and improving cross-modal homography estimation.