<p>To address the issue that large variations in appearance and scale among images of the same class lead to poor generalization of segmentation models on unseen images, a few-shot semantic segmentation method based on dual-branch feature continual learning is proposed. First, a pair of shared-weight backbone networks map the support and query images into a deep feature space. The ground-truth mask of the support image is then used to separate its encoded features into foreground object features and background features. Next, the CLIP-encoded textual features are used to disentangle the mixed query features. The separated dual-branch foreground and background features are subsequently fused and aligned, enabling continual learning between branches through the target task. Finally, mask average pooling is applied to the fused foreground and background features to generate corresponding prototypes, and a parameter-free metric matching is performed to match query features with the prototype set on a per-pixel basis. Experiments on the PASCAL-5<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(^{\textit{i}}\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mi mathvariant="italic">i</mi> </mmultiscripts> </math></EquationSource> </InlineEquation> and COCO-20<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(^{\textit{i}}\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mi mathvariant="italic">i</mi> </mmultiscripts> </math></EquationSource> </InlineEquation> datasets under 1-shot and 5-shot settings demonstrate that the proposed method achieves competitive segmentation perfor-mance, comparable to state-of-the-art few-shot semantic segmentation methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual-branch feature continual learning for few-shot semantic segmentation

  • Fuqiang Chen,
  • Huifang Zhao,
  • Guisheng Tan,
  • Yijin Shi

摘要

To address the issue that large variations in appearance and scale among images of the same class lead to poor generalization of segmentation models on unseen images, a few-shot semantic segmentation method based on dual-branch feature continual learning is proposed. First, a pair of shared-weight backbone networks map the support and query images into a deep feature space. The ground-truth mask of the support image is then used to separate its encoded features into foreground object features and background features. Next, the CLIP-encoded textual features are used to disentangle the mixed query features. The separated dual-branch foreground and background features are subsequently fused and aligned, enabling continual learning between branches through the target task. Finally, mask average pooling is applied to the fused foreground and background features to generate corresponding prototypes, and a parameter-free metric matching is performed to match query features with the prototype set on a per-pixel basis. Experiments on the PASCAL-5 \(^{\textit{i}}\) i and COCO-20 \(^{\textit{i}}\) i datasets under 1-shot and 5-shot settings demonstrate that the proposed method achieves competitive segmentation perfor-mance, comparable to state-of-the-art few-shot semantic segmentation methods.