Traditional mitigation approaches for backdoored neural networks, such as unlearning and finetuning, face significant challenges when applied to self-supervised learning (SSL) models. Unlearning through neuron pruning often causes catastrophic loss of legitimate features, while finetuning fails to effectively update backdoor neurons due to their dormant nature during clean sample processing. Both approaches distort the learned feature distributions due to the lack of label supervision in SSL settings. We propose Inversion Triplet, a novel mitigation method that leverages contrastive learning principles to overcome these limitations. Our key insight is that by treating clean samples as positive pairs and backdoored samples as negative pairs in a triplet loss formulation, we can isolate and eliminate poison feature clusters while preserving the semantic structure of clean representations. This approach requires neither white-box model access nor label supervision, making it practical for real-world deployment. Extensive experiments demonstrate that Inversion Triplet reduces attack success rates compared to existing methods while maintaining best model utility on clean data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Inversion Triplet - A Contrastive Backdoor Mitigation Method for Self-Supervised Vision Encoders

  • Hiep Vo,
  • Zhiyi Tian,
  • Chenhan Zhang,
  • Xi James Zheng,
  • Shui Yu

摘要

Traditional mitigation approaches for backdoored neural networks, such as unlearning and finetuning, face significant challenges when applied to self-supervised learning (SSL) models. Unlearning through neuron pruning often causes catastrophic loss of legitimate features, while finetuning fails to effectively update backdoor neurons due to their dormant nature during clean sample processing. Both approaches distort the learned feature distributions due to the lack of label supervision in SSL settings. We propose Inversion Triplet, a novel mitigation method that leverages contrastive learning principles to overcome these limitations. Our key insight is that by treating clean samples as positive pairs and backdoored samples as negative pairs in a triplet loss formulation, we can isolate and eliminate poison feature clusters while preserving the semantic structure of clean representations. This approach requires neither white-box model access nor label supervision, making it practical for real-world deployment. Extensive experiments demonstrate that Inversion Triplet reduces attack success rates compared to existing methods while maintaining best model utility on clean data.