VFIQ: A Novel Model of ViT-FSIMc Hybrid Siamese Network for Image Quality Assessment

The Image Quality Assessment (IQA) is to measure how humans perceive the quality of images. In this paper, we propose a new model named for VFIQ – a ViT-FSIMc Hybrid Siamese Network for Full Reference IQA – that combines signal processing and leaning-based approaches, the two categories of IQA algorithms. Specifically, we design a hybrid Siamese network that leverages the Vision Transformer (ViT) and the feature similarity index measurement (FSIMc). To evaluate the performance of the proposed VFIQ model, we first pre-train the ViT module on the PIPAL dataset, and then evaluate our VFIQ model on several popular benchmark datasets including TID2008, TID2013, and LIVE. The experiment results show that our VFIQ model outperforms the state-of-the-art IQA models in the commonly used correlation metrics of PLCC, KRCC, and SRCC. We also demonstrate the usefulness of our VFIQ model in different vision tasks, such as image recovery and generative model evaluation.