<p>There is a crucial problem posing great challenges to the image quality assessment (IQA), that is, how to accurately regress the visual quality score of an entire image from its patches. The vast majority of existing patch-based IQA methods treat each patch independently. In this paper, we innovatively enable inter-patch message passing (MP) for the proposed IQA via graph convolutional networks (IQG). The patches are embedded into the graph by treating the low-dimensional vector representation of each patch as a node and the inter-patch intrinsic correlation as an edge. Since the intrinsic correlation is not directly available, an adaptive edge generator is proposed to adaptively construct the directed weighted edges by separately obtaining the patch-connected mask and the edge weights. To mitigate the overfitting that may occur when adaptive MP is enabled, we attach an embedding approach that creates the undirected unweighted edge between any two patches to enable each node in the graph to connect to every other node, thus passing the information that otherwise would be neglected. Extensive experiments demonstrate the state-of-the-art performance of our proposed IQG in complete scenarios, including full-reference and no-reference IQA tasks on benchmark IQA databases.</p>