Multi-modal Metaphor Explanation: A Dataset and Benchmark
摘要
Creative images are images rich in implicit semantic information formed through artistic processing to express specific intentions such as persuasion, exhortation, and warning. Understanding and interpreting the implied intentions of creative images which usually accompany with metaphors is very valuable but challenging. In this paper, we introduce a Multi-modal Metaphor Explanation (MME) task: Given a creative image with a related condensed slogan, MME aims to generate the explanation to reveal the delicate metaphor. Different from image captioning which understands and describes the literal semantics of images, explaining multi-modal metaphor requires correctly recognizing semantic associations between the source and target domains, understanding aesthetic conception from concrete visual operations or artistic designs, and further interpreting the implicit intentions. To support the research on MME, we develop CIME, a new dataset with 3753 high-quality creative images and related condensed slogans. We build an extensive benchmark and set up rigorous evaluation metrics for this new research problem. Experimental results on the automatic evaluation metrics demonstrate the superiority of our dataset quality. Our dataset is available at https://github.com/VILANLab/Metaphor-CIME