Graph Neural Networks (GNNs) have shown great potential in visual tasks, yet they face challenges in effectively constructing and processing graphs. Vision GNN (ViG) was developed to tackle these issues by segmenting images into patches treated as nodes, with edges formed by connecting the nearest semantic neighbors. However, relying solely on semantic information for graph construction confines itself to a dispersed distribution of object neighbors, leading to inadequate graph processing. To address this issue, we propose Vision DualGNN(VDG), a novel dual graph neural network architecture that leverages both spatial and semantic information to construct and process graph representation of images. We apply a node encoder that transforms image patches into expressive node features. Additionally, we implement a dual-stream GNN that operates on both a spatial graph and a semantic graph. The spatial graph serves as a constraint for the semantic graph, enhancing the node features with spatial awareness. To verify the validity of our architecture, we have conducted our experiments on the ImageNet and CIFAR-100 datasets. And achieved state-of-the-art performance compared to other baseline models.