<p>With the rapid advancement of imaging technologies, multi-modal image matching has become a prominent research focus in the field of computer vision. Images captured by different sensors or imaging modalities often contain complementary information, which presents both opportunities and challenges for accurate and robust image matching. Although numerous studies have aimed to improve matching performance, existing methods often suffer from insufficient descriptor extraction and a lack of clear semantic guidance during the matching process. To address these challenges, we propose a multi-modal image matching scheme based on regional semantic guidance (RSGM). Our core innovation is the regional semantic information guiding matching module (SGM), which introduces a novel “segment-then-match” paradigm inspired by human cognition. This module constrains the matching domain from the entire image to corresponding semantic guidance regions, thereby reducing the computational cost and improving the accuracy and robustness of the matching process. To support this semantic-guided matching process, we further introduce two novel descriptor enhancement modules: the similarity information fusion module (SIFM) and the cross image neighborhood pixel information aggregation module (CNIM). These modules enrich the feature descriptors by fusing explicit global similarity structures and cross-image neighborhood information, respectively, providing robust inputs for the SGM stage. Extensive experimental results demonstrate that our proposed RSGM framework achieves superior performance on multiple multi-modal image datasets. The code for RSGM is available at: <a href="https://github.com/LiaoYun0x0/RSGM">https://github.com/LiaoYun0x0/RSGM</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

RSGM: a multi-modal image matching scheme guided by regional semantics

  • Fangwei Jin,
  • Yun Liao,
  • Junhui Liu,
  • ZongXiao Hu,
  • Xu Qian,
  • YunPeng Li,
  • RongRui Teng,
  • Qing Duan

摘要

With the rapid advancement of imaging technologies, multi-modal image matching has become a prominent research focus in the field of computer vision. Images captured by different sensors or imaging modalities often contain complementary information, which presents both opportunities and challenges for accurate and robust image matching. Although numerous studies have aimed to improve matching performance, existing methods often suffer from insufficient descriptor extraction and a lack of clear semantic guidance during the matching process. To address these challenges, we propose a multi-modal image matching scheme based on regional semantic guidance (RSGM). Our core innovation is the regional semantic information guiding matching module (SGM), which introduces a novel “segment-then-match” paradigm inspired by human cognition. This module constrains the matching domain from the entire image to corresponding semantic guidance regions, thereby reducing the computational cost and improving the accuracy and robustness of the matching process. To support this semantic-guided matching process, we further introduce two novel descriptor enhancement modules: the similarity information fusion module (SIFM) and the cross image neighborhood pixel information aggregation module (CNIM). These modules enrich the feature descriptors by fusing explicit global similarity structures and cross-image neighborhood information, respectively, providing robust inputs for the SGM stage. Extensive experimental results demonstrate that our proposed RSGM framework achieves superior performance on multiple multi-modal image datasets. The code for RSGM is available at: https://github.com/LiaoYun0x0/RSGM.