Image Quality Assessment Method Based on Cross-Modal

Deep learning methods have achieved remarkable results in the direction of image quality assessment tasks. However, most of the related studies focus only on image unimodal, ignoring the potential advantages that come with the development of cross-modal techniques. Cross-modal models have implied a wealth of information, which provides new research directions and possibilities in the field of image quality assessment. In this paper, the feasibility of cross-modal models in image quality assessment is first explored for the image quality binary classification task. Subsequently, the optimization prompting method is combined with the tuning of the image encoder in the cross-modal model so that the cross-modal model can be used for image quality assessment scoring. To verify the feasibility of the cross-modal model on the image quality assessment task, an empirical analysis was conducted on the binary image quality dataset PQD, and it was found that the F1 score improved by 18% over the baseline model. Further, we propose an adaptive cross-modal image quality assessment method AC-IQA. On the image quality scoring dataset, compared with the previous optimal methods, AC-IQA improves the PLCC and SROCC metrics on the TID2013 dataset by 5.5% and 9.5%, respectively, and on the KADID dataset by 6.2% and 5.2%.