A Spatiotemporal Comprehensive Graph-Based Learning for GIF Sentiment Analysis

As social media has grown, more and more people are using short videos in social media applications to share their thoughts and feelings. On the other hand, the semantic gap problem and the sequence-based sentiment understanding problem make sentiment detection in short videos extremely difficult. From the existing work it has been observed that the temporal feature of a GIF has not been considered with due importance. We propose a spatiotemporal comprehensive graph-based strategy to bridge the gap between spatial and temporal features of a GIF. In our proposed method the region in the temporal domain is considered with the region of spatial domain. Using intra-frame and inter-frame feature map, the weight matrix is learned without prior knowledge, with the help of the combination of the image frames and the feature extractor. In order to demonstrate the effectiveness of the suggested spatiotemporal comprehensive graph-based strategy, we conducted extensive experiments with a variety of GIF.