Bilinear Pooling of Transformer Embeddings for Blind Image Quality Assessment

Blind image quality assessment finds its practical usage in real-world applications where image distortions are more complex than computer generated synthetic distortions, but high-quality images are not available for reference. In the past decade, research in blind quality prediction has advanced tremendously thanks to the success of convolutional neural networks. However, it is far from human-like performance and remains a challenging research problem. For the first time, this paper investigates the potential of imagenet pre-trained Vision Transformer, a new generation architecture for image understanding, in providing better quality aware features. This paper proposed BPTIQ, a method that leverages multi-level transformer embeddings with bilinear feature pooling and non-monotonic error regularization for blind quality assessment of authentic distortions. The effectiveness of the proposed method was evaluated with four IQA databases with authentic distortions. Experimental outcomes and ablation studies show that the performance of BPTIQ is competitive with nine state-of-the-art IQA methods in comparison that mainly utilized pre-trained convolutional neural networks for feature extraction. BPTIQ performed the best over two of the four single databases and demonstrated a more robust cross-database generalization capability.