<p>Redundant weights bring security risks to transfer learning, how to remove redundant weights effectively is an important research topic in transfer learning. Currently, redundant weights are removed mainly through pruning algorithms designed by correlating student data with teacher model nodes. However, these methods do not consider the ability of nodes to resist disturbance signals and malicious injection, limiting the application of transfer learning. To deal with this, this paper proposes an immune sparse therapy for transfer learning (ISTL) inspired by Specific Immunotherapy in the biomedical field. The ISTL treats the teacher model as a paternal gene, the student model as an allergy patient, the adversarial attack as an allergen, and the misclassification of the model after the attack as an allergic reaction. Firstly, ISTL utilizes the teacher model (paternal gene) to construct an Immune Search Space (treatment options library) and take the fully activated state (Immune Overreaction) as the starting point for training (treatment). Meanwhile, adversarial perturbations (antigens) are added to the student samples (non-allergens) to generate specialized vaccines for subsequent training (treatment). Secondly, to evaluate the model's treatment effect, this paper proposes Integrated Multi-Criteria Loss (IMCL), which fuses each parameter index in each round of processing. Finally, according to the treatment effect, ISTL adjusts the treatment plan by activating or deactivating nodes to achieve better safety performance while maintaining similar accuracy. Experiments show that, compared with the 8 latest models, ISTL improves the model's ability to resist adversarial attacks and backdoor attacks by 75.72% and 63.4% on 8 datasets while maintaining accuracy.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A redundant weight removal method inspired by specific immunotherapy for boosting transfer learning security

  • Qing Huang,
  • Hongli Deng,
  • Junxiang Wang,
  • Tao Yang,
  • Bochuan Zheng

摘要

Redundant weights bring security risks to transfer learning, how to remove redundant weights effectively is an important research topic in transfer learning. Currently, redundant weights are removed mainly through pruning algorithms designed by correlating student data with teacher model nodes. However, these methods do not consider the ability of nodes to resist disturbance signals and malicious injection, limiting the application of transfer learning. To deal with this, this paper proposes an immune sparse therapy for transfer learning (ISTL) inspired by Specific Immunotherapy in the biomedical field. The ISTL treats the teacher model as a paternal gene, the student model as an allergy patient, the adversarial attack as an allergen, and the misclassification of the model after the attack as an allergic reaction. Firstly, ISTL utilizes the teacher model (paternal gene) to construct an Immune Search Space (treatment options library) and take the fully activated state (Immune Overreaction) as the starting point for training (treatment). Meanwhile, adversarial perturbations (antigens) are added to the student samples (non-allergens) to generate specialized vaccines for subsequent training (treatment). Secondly, to evaluate the model's treatment effect, this paper proposes Integrated Multi-Criteria Loss (IMCL), which fuses each parameter index in each round of processing. Finally, according to the treatment effect, ISTL adjusts the treatment plan by activating or deactivating nodes to achieve better safety performance while maintaining similar accuracy. Experiments show that, compared with the 8 latest models, ISTL improves the model's ability to resist adversarial attacks and backdoor attacks by 75.72% and 63.4% on 8 datasets while maintaining accuracy.