Visual Sentiment Analysis with a VR Sentiment Dataset on Omni-Directional Images

Visual content can affect viewer’s emotions, which makes sentiment analysis of visual content more and more concerned. Sentiment analysis on omni-directional images plays an important role in virtual reality (VR) applications such as user behaviour prediction, game scene modelling, psychotherapy, etc. However, due to the serious lack of validated VR emotional datasets, the research progress of sentiment analysis in VR is very slow. In this paper, firstly, we build a VR sentiment dataset containing 1,140 emotion-eliciting omni-directional images. Secondly, a pyramidal dual attention network is proposed to analyse the sentiment task. According to the characteristics of omni-directional images, this network utilizes the dual attention module to capture emotion-eliciting regions and adaptively establish the connection between them. Furthermore, objects of different scales have different contributions to evoke emotions. Therefore, the pyramidal feature hierarchy can analyse objects with different complexity by using multi-layer visual features. Finally, quantitative and qualitative experiments on the self-established dataset illustrate that the proposed network can effectively predict the regions that elicit emotions.