Countering Adversarial Attacks with Multimodal Image Fusion
摘要
With the rise of generative AI, multimodal perception plays an increasingly critical role in building robust systems. Although multimodal image fusion holds great potential for defending against adversarial perturbations, current methods remain vulnerable due to two key limitations: the lack of attack-aware mechanisms during fusion and the absence of a unified, multi-stage defense strategy throughout the fusion pipeline. To address this issue, we propose an attack-aware multimodal fusion model that enhances robustness by integrating a multi-layered defense strategy. Specifically, a pre-trained denoising autoencoder is introduced during feature extraction to suppress input noise and produce clearer, more robust representations. In the fusion stage, an attack-aware fusion module dynamically estimates the perturbation level of each modality and adjusts fusion weights accordingly to down-weight unreliable inputs. During training, adversarial training is applied to further improve the model’s resistance to various types of attacks. These components work together as a coordinated multi-layer defense system to improve overall model stability. Experimental results show that the proposed approach significantly outperforms existing baselines in terms of robustness across core tasks such as image captioning, semantic segmentation, and object detection, even under adversarial conditions.