ICPE-FAS: Instance and Category Prompts Engineering for Generalizable Face Anti-Spoofing
摘要
Domain generalization (DG) based Face Anti-Spoofing (FAS) aims to improve the model’s performance on unseen domains. Existing methods either rely on domain labels to align domain-invariant feature spaces or disentangle generalizable features from the whole sample, which inevitably leads to the distortion of semantic structures and achieves limited generalization. In this work, we make use of large-scale VLMs like CLIP and leverage the textual features to dynamically adjust the classifier’s weights for exploring generalizable visual features. Specifically, we propose a novel CLIP with Instance and Category Prompts Engineering for DG FAS (ICPE-FAS), which includes two text branches: (1) Instance-wise Content Prompt Learning (ICPL) to generate instance-prompts consisting of two lightweight transformers, namely Content Q-Former (CQF) and Style Q-Former (SQF), to learn the different semantic prompts conditioned on content and style features by using a set of learnable query vectors, respectively. The generalizable content prompt can be learned by two innovations: (a) A Prompt-Text Matched (PTM) supervision is introduced to ensure CQF learns a visual representation that is most informative of the content description. (b) A Diversified Style Prompt (DSP) technology is proposed to diversify the learning of style prompts by mixing feature statistics between instance-specific styles. Finally, the learned text features modulate visual features to generalization through the designed Prompt Modulation (PM). (2) Category-wise Segmented Prompt Learning (CSPL) to generate Domain-agnostic Prompt (DaP) and Domain-specific Prompt (DsP), which can serve as weights for the classifier to accurately suppress domain-related signals. Extensive experiments show that the ICPE-FAS is effective and outperforms the state-of-the-art methods on several cross-domain benchmarks.