Multi-depth Fusion Transformer and Batch Piecewise Loss for Visual Sentiment Analysis

Visual sentiment analysis is an important research direction in the field of affective computing, which is of great significance in user behavior prediction, visual scene construction, etc. In order to address the issue of low efficiency in high-level semantic perception of CNNs, in this paper, the Transformer architecture is introduced into the task of visual sentiment analysis, proposing the Multi-depth Fusion Transformer (MFT) model that allows nesting different architectures. Additionally, addressing the problem of imbalanced data samples and varying learning difficulties in visual sentiment analysis datasets, this paper proposes a batch piecewise loss suitable for visual sentiment binary classification tasks. Experimental results on multiple datasets demonstrate that the proposed method outperforms existing approaches. Visualization experiments comparing CNN and Transformer-based approaches demonstrate the effectiveness of the proposed method in semantic perception.