Towards Efficient and Secure Multimodal Misinformation Detection
摘要
The rapid development of AI-generated content (AIGC) has heightened the threat of multimodal misinformation, posing significant security concerns. Existing misinformation detection methods have not addressed privacy concerns or have only considered the privacy of single-modal data, overlooking the semantic correlation privacy among multimodal data. In this paper, we propose a cloud-edge-end collaboration-based privacy-preserving multimodal misinformation detection (CPMD) scheme, safeguarding the correlation privacy of multimodal data while efficiently detecting misinformation. To achieve this, we introduce a local differential privacy-based multimodal Fourier dynamic protection (MFDP) scheme, thereby achieving efficient multimodal fusion and semantic correlation protection. We also design a multi-head spatiotemporal correlation attention-based large model pre-training protection (MSLP) method and DP-based Gradient Correlation Protection (DPGC) method, which ensure high-utility gradient correlation protection. Furthermore, we utilize a multimodal misinformation detection transformer (MMDT) to perform high accuracy recognition. Extensive experiments demonstrate that, compared to the HAMMER method, our CPMD method improves the accuracy of multimodal misinformation detection by 5.08%.