Image Attribute Completion: A Novel Task for Multi-modal Knowledge Graph
摘要
Multi-modal knowledge graphs (MMKGs) present distinct advantages over unimodal knowledge graphs by leveraging diverse modal attributes for more comprehensive entity descriptions. Most existing studies on multi-modal knowledge graph completion focus on link prediction and numerical attribute completion, neglecting another significant issue, i.e., the incompleteness of image attribute. To fill the gap, this paper formulates a novel task Image Attribute Completion, and develops a dedicated model entitled MAGIC (Multi-modal Assisted imaGe attrIbutes Completion). Specifically, the model consists of two main components: (1) The module EEM is designed to Encode Entity with Different Modalities (i.e., structural information, discrete attributes, text, and images) by employing distinct encoders to learn modality-specific embeddings. Then, these representations are dynamically fused by calculating the entropy of each modality. (2) Based on the fused embeddings of entities, the module EIA is introduced to perform Entity-Image Alignment, which is achieved through a two-step process. First, we calculate the similarity between entities and candidate images. Second, two loss functions are combined to simultaneously enhance modality alignment and classification accuracy. The experiments conducted on three real-world datasets demonstrate the superiority of our proposed model.