Image registration, while essential for aligning multi-modal data, often introduces invalid boundary regions known as unknown borders. These artifacts degrade visual quality and disrupt structural integrity, ultimately affecting the reliability of downstream analysis. Although many approaches have been proposed for border completion, they commonly rely on large-scale annotated datasets and struggle to integrate local features with global context, especially in multi-modal scenarios. To address these challenges, we introduce BorderFormer, a self-supervised method for unknown border completion that eliminates the need for manual annotations while fully leveraging complementary information across modalities. To overcome the difficulty of acquiring ground-truth labels, we first design a Self-supervised Data Generation (SDG) module that constructs diverse training pairs by extracting valid content and recombining it through a concatenation-cropping strategy. Building upon this, BorderFormer incorporates a Multi-modal Feature Aggregation (MFA) module to extract and align information from different modalities using a modality-guided dual-branch encoder and multi-scale fusion. Finally, a Hierarchical Neighborhood Attention (HNA) module enhances structural coherence by capturing fine-grained details and modeling long-range dependencies across border regions. Experiments on various multi-modal datasets show that BorderFormer consistently outperforms existing approaches in PSNR and SSIM. Qualitative results further demonstrate its ability to produce visually natural and structurally consistent completions in complex registration scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Beyond Registration: Self-supervised Unknown Border Completion

  • Xiaokai Bai,
  • Jun Ma,
  • Leyuan Yu,
  • Runmin Zhang,
  • Hui-Liang Shen,
  • Beinan Yu,
  • Si-Yuan Cao

摘要

Image registration, while essential for aligning multi-modal data, often introduces invalid boundary regions known as unknown borders. These artifacts degrade visual quality and disrupt structural integrity, ultimately affecting the reliability of downstream analysis. Although many approaches have been proposed for border completion, they commonly rely on large-scale annotated datasets and struggle to integrate local features with global context, especially in multi-modal scenarios. To address these challenges, we introduce BorderFormer, a self-supervised method for unknown border completion that eliminates the need for manual annotations while fully leveraging complementary information across modalities. To overcome the difficulty of acquiring ground-truth labels, we first design a Self-supervised Data Generation (SDG) module that constructs diverse training pairs by extracting valid content and recombining it through a concatenation-cropping strategy. Building upon this, BorderFormer incorporates a Multi-modal Feature Aggregation (MFA) module to extract and align information from different modalities using a modality-guided dual-branch encoder and multi-scale fusion. Finally, a Hierarchical Neighborhood Attention (HNA) module enhances structural coherence by capturing fine-grained details and modeling long-range dependencies across border regions. Experiments on various multi-modal datasets show that BorderFormer consistently outperforms existing approaches in PSNR and SSIM. Qualitative results further demonstrate its ability to produce visually natural and structurally consistent completions in complex registration scenarios.