Predicting propagation effects of tweets via a multimodal feature fusion model based on co-attention mechanism
摘要
Twitter is the main source of information for most people abroad, and predicting the propagation effects of tweets can assist content creators in strategically planning their content to maximize audience reach and engagement. Tweets typically contain multimodal information, including both text and images. However, existing studies often consider only text features or simply concatenate text and image features, which limits their ability to fully capture the rich characteristics of tweets and achieve ideal prediction performance. To address this issue, this paper proposes a multimodal feature fusion approach to predict the propagation effects of tweets. This approach uses the BERT model to obtain word embeddings of tweets and employs TextCNN to extract the semantic features of text in tweets. Meanwhile, the semantic features of images in tweets are extracted using the ResNet-50 model. A multimodal fusion model based on a co-attention mechanism is then introduced to integrate the textual and visual semantic features by applying text-guided visual attention and image-guided textual attention, thereby capturing richer semantic information. After obtaining the fused multimodal features, these features are concatenated with auxiliary features (e.g., the number of followers of the author, the length of text, and the sentiment features of text) and fed into a classifier to predict the propagation effects of tweets. Comparative experiments demonstrate that the proposed approach can effectively predict the propagation effects of tweets.