<p>Domain generalization (DG) based Face Anti-Spoofing (FAS) aims to improve the model’s performance on unseen domains. Existing methods either rely on domain labels to align domain-invariant feature spaces or disentangle generalizable features from the whole sample, which inevitably leads to the distortion of semantic structures and achieves limited generalization. In this work, we make use of large-scale VLMs like CLIP and leverage the textual features to dynamically adjust the classifier’s weights for exploring generalizable visual features. Specifically, we propose a novel CLIP with Instance and Category Prompts Engineering for DG FAS (<b>ICPE-FAS</b>), which includes two text branches: (1) Instance-wise Content Prompt Learning (<b>ICPL</b>) to generate instance-prompts consisting of two lightweight transformers, namely Content Q-Former (CQF) and Style Q-Former (SQF), to learn the different semantic prompts conditioned on content and style features by using a set of learnable query vectors, respectively. The generalizable content prompt can be learned by two innovations: (a) A Prompt-Text Matched (<b>PTM</b>) supervision is introduced to ensure CQF learns a visual representation that is most informative of the content description. (b) A Diversified Style Prompt (<b>DSP</b>) technology is proposed to diversify the learning of style prompts by mixing feature statistics between instance-specific styles. Finally, the learned text features modulate visual features to generalization through the designed Prompt Modulation (<b>PM</b>). (2) Category-wise Segmented Prompt Learning (<b>CSPL</b>) to generate Domain-agnostic Prompt (<b>DaP</b>) and Domain-specific Prompt (<b>DsP</b>), which can serve as weights for the classifier to accurately suppress domain-related signals. Extensive experiments show that the ICPE-FAS is effective and outperforms the state-of-the-art methods on several cross-domain benchmarks.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ICPE-FAS: Instance and Category Prompts Engineering for Generalizable Face Anti-Spoofing

  • Ajian Liu,
  • Xun Lin,
  • Hui Ma,
  • Xinxing Yu,
  • Jiabao Guo,
  • Zitong Yu,
  • Jun Wan,
  • Zhanchuan Cai,
  • Zhen Lei,
  • Yanyan Liang

摘要

Domain generalization (DG) based Face Anti-Spoofing (FAS) aims to improve the model’s performance on unseen domains. Existing methods either rely on domain labels to align domain-invariant feature spaces or disentangle generalizable features from the whole sample, which inevitably leads to the distortion of semantic structures and achieves limited generalization. In this work, we make use of large-scale VLMs like CLIP and leverage the textual features to dynamically adjust the classifier’s weights for exploring generalizable visual features. Specifically, we propose a novel CLIP with Instance and Category Prompts Engineering for DG FAS (ICPE-FAS), which includes two text branches: (1) Instance-wise Content Prompt Learning (ICPL) to generate instance-prompts consisting of two lightweight transformers, namely Content Q-Former (CQF) and Style Q-Former (SQF), to learn the different semantic prompts conditioned on content and style features by using a set of learnable query vectors, respectively. The generalizable content prompt can be learned by two innovations: (a) A Prompt-Text Matched (PTM) supervision is introduced to ensure CQF learns a visual representation that is most informative of the content description. (b) A Diversified Style Prompt (DSP) technology is proposed to diversify the learning of style prompts by mixing feature statistics between instance-specific styles. Finally, the learned text features modulate visual features to generalization through the designed Prompt Modulation (PM). (2) Category-wise Segmented Prompt Learning (CSPL) to generate Domain-agnostic Prompt (DaP) and Domain-specific Prompt (DsP), which can serve as weights for the classifier to accurately suppress domain-related signals. Extensive experiments show that the ICPE-FAS is effective and outperforms the state-of-the-art methods on several cross-domain benchmarks.