Pseudo-global strategy-based visual comfort assessment considering attention mechanism

Assessing the comfort of stereo images contributes significantly to crafting immersive stereo scenes, thereby enriching the viewer’s perceptual experience. However, deep learning-based visual comfort assessment (VCA) has encountered challenges due to data deficiency. To address this problem and maximize the potential of deep learning in the VCA task, this paper proposes a pseudo-global strategy-based convolutional neural network (CNN), considering the attention mechanism. Our data augmentation method utilizes random cropping and permutation, coupled with a pseudo-global strategy that fuses multi-region local features as pseudo-global features to substitute global features, effectively expanding databases while aligning input patches and labels during training. We also introduce attention mechanisms to focus on the different impacts of disparities in various regions on the overall comfort of a stereo image. Specifically, dilated spatial attention and channel self-attention are designed in the local and pseudo-global feature extraction stages, respectively, simulating the saliency of human perception. Experimental results show that the proposed method is superior to the state-of-the-art VCA approaches and has excellent generalization ability.