Scholarlink

<p>This paper proposes an H-type Bidirectional Alignment Network for text-guided image segmentation, enabling efficient and accurate cross-modal feature fusion. For visual encoding, a 12-layer Vision Mamba with four stages is adopted. It captures long-range dependencies via a Selective State Space Model (SSM) and, at the same time, leverages local convolution and residual structures to enhance boundary and detail information—thereby reducing computational complexity when processing high-resolution images. The text encoder is based on the Qwen model and employs a strategy of freezing the bottom layers while fine-tuning the upper layers, aiming to obtain semantic representations and adapt to referential expressions. The cross-modal alignment module utilizes Q-Former to construct learnable query vectors: the forward path accomplishes the text-to-image segmentation task, while the backward path reconstructs the attention distribution of the image over the text. This bidirectional supervision mechanism is thus realized to constrain cross-modal consistency. The multi-scale decoder fuses visual and aligned features, and supports the model in gradually optimizing segmentation results through an interactive iterative mechanism. Experiments on the RefCOCO, RefCOCO+, and RefCOCOg datasets verify the effectiveness of the proposed method, which demonstrates performance improvements compared to existing approaches.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Interactive text-guided image segmentation via vision Mamba and large language models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月18日

查看全文
  • Yao Meng,
  • Haochen Sun,
  • Wei Jiang

摘要

This paper proposes an H-type Bidirectional Alignment Network for text-guided image segmentation, enabling efficient and accurate cross-modal feature fusion. For visual encoding, a 12-layer Vision Mamba with four stages is adopted. It captures long-range dependencies via a Selective State Space Model (SSM) and, at the same time, leverages local convolution and residual structures to enhance boundary and detail information—thereby reducing computational complexity when processing high-resolution images. The text encoder is based on the Qwen model and employs a strategy of freezing the bottom layers while fine-tuning the upper layers, aiming to obtain semantic representations and adapt to referential expressions. The cross-modal alignment module utilizes Q-Former to construct learnable query vectors: the forward path accomplishes the text-to-image segmentation task, while the backward path reconstructs the attention distribution of the image over the text. This bidirectional supervision mechanism is thus realized to constrain cross-modal consistency. The multi-scale decoder fuses visual and aligned features, and supports the model in gradually optimizing segmentation results through an interactive iterative mechanism. Experiments on the RefCOCO, RefCOCO+, and RefCOCOg datasets verify the effectiveness of the proposed method, which demonstrates performance improvements compared to existing approaches.