MEIG: An Interactive Attention-Based Generative Model for Multimodal Event Extraction
摘要
Multimodal event extraction is a key area of information extraction, focusing on retrieving structured events from multimodal data sources. Traditional methods usually utilize shallow image information to assist in event extraction, which results in the loss of associated information between text and images in multimodal instances. In this paper, we propose MEIG, a generative multimodal event extraction model that focuses on deep related information between images and text and can extract structured events directly from the instances. Given a pair of image and text, MEIG uses an interactive attention mechanism to obtain overall image features and object features closely related to the text content in the image, enhancing the representation of text features and getting structured events through constrained decoding. Additionally, due to the lack of multimodal event extraction dataset, we constructed a naturally aligned dataset multimodal event extraction dataset. Experimental results on this dataset demonstrate that the model proposed in this paper achieves good performance.