AMNN: Uniting Attention Mechanism and Adversarial Contrastive Learning for Adaptive Multimodal Knowledge Graph Completion
摘要
Multimodal Knowledge Graph Completion (MMKGC) is dedicated to addressing the issue of missing link prediction in knowledge graphs. Its core methodology lies in the synergistic integration of multi-source heterogeneous information, including structured associations, visual representations, and textual semantics, thereby constructing discriminative models capable of deep joint reasoning. However, existing MMKGC methods commonly suffer from insufficient fine-grained cross-modal alignment when fusing multimodal information. To address this issue, this paper proposes an Adaptive Multi-modal Neural Network framework, named AMNN. Specifically, AMNN employs a Bidirectional Alternating Attention Mechanism to enhance the fine-grained association process from local to global levels across semantic units of different modalities, achieving precise mapping between image regions and textual phrases. Simultaneously, an adversarial neural network utilizes attention weights to generate modality-aware adversarial samples that augment contrastive learning, thereby improving the robustness of the MMKGC model. These two components engage in closed-loop collaborative training: the Bidirectional Alternating Attention Mechanism directs the adversarial contrastive learning module to generate modality-aware adversarial samples, while the adversarial contrastive learning module, through its contrastive loss, inversely optimizes the attention distribution of the Bidirectional Alternating Attention Mechanism. Experiments conducted on two benchmark datasets demonstrate that AMNN achieves significant improvements: MRR increases by 5.1% and 2.0%, and Hit@1 increases by 5.2% and 2.5% on the DB15K and MKG-W datasets, respectively. These results surpass the performance of 14 existing state-of-the-art models, validating the effectiveness of the proposed framework.