Scholarlink

<p>Recent advances in Large Language Models (LLMs) have paved the way for the emergence of Multimodal and Multitask LLMs (MMLLMs). While poisoning attacks are well studied in traditional AI models, they have not been characterized for MMLLMs. This paper focuses on a MMLLM-specific threat, dubbed <i>cross-contamination</i>, by which one task may be indirectly affected by the poison inserted in other tasks. In this vein, this paper addresses representative data poisoning techniques (label flipping and backdooring) across different text-, image- and video-based tasks. We analyse the effect on three representative models, namely OFA, LLaMa Vision 3.2 and Qwen2 Visual. The size of the dataset, the poison budget, the number of epochs and the generation strategy are analysed. Results confirm that poisoning can spread through unrelated tasks while preserving the overall model performance. Interestingly, attack effectivity rates above 99% can be achieved with just 5% of poison. The level of stealthiness depends on the type of attack, the targeted task and the model. We have also found a <i>surprise effect</i> by which a higher number of epochs do not always lead to a greater spread of the poison. Lastly, the choice of generation strategies has led to differences beyond 70% in the attack success rate.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Characterizing Cross-Contamination on Multitask Multimodal Large Language Models

  • 期刊论文

  • 发表日期：

    2025年12月8日

查看全文
  • Luis Ibañez-Lissen,
  • Lorena González-Manzano,
  • José M. de Fuentes,
  • Joaquin Garcia-Alfaro

摘要

Recent advances in Large Language Models (LLMs) have paved the way for the emergence of Multimodal and Multitask LLMs (MMLLMs). While poisoning attacks are well studied in traditional AI models, they have not been characterized for MMLLMs. This paper focuses on a MMLLM-specific threat, dubbed cross-contamination, by which one task may be indirectly affected by the poison inserted in other tasks. In this vein, this paper addresses representative data poisoning techniques (label flipping and backdooring) across different text-, image- and video-based tasks. We analyse the effect on three representative models, namely OFA, LLaMa Vision 3.2 and Qwen2 Visual. The size of the dataset, the poison budget, the number of epochs and the generation strategy are analysed. Results confirm that poisoning can spread through unrelated tasks while preserving the overall model performance. Interestingly, attack effectivity rates above 99% can be achieved with just 5% of poison. The level of stealthiness depends on the type of attack, the targeted task and the model. We have also found a surprise effect by which a higher number of epochs do not always lead to a greater spread of the poison. Lastly, the choice of generation strategies has led to differences beyond 70% in the attack success rate.