Currently, vein recognition technology is mainly limited to the hand (finger, dorsal, and palm). In comparison, the arm region has more vein textures and richer feature information. It shows a broader application potential. Traditional vein feature matching techniques have poor robustness. While the method based on convolutional neural networks (CNNs) can take into account the global features of the image, it is easy to ignore the structural relationship between vein features. Recognition accuracy is especially low in some small-sample datasets. In this study, we propose an arm vein recognition method based on multi-hop graph convolutional networks (GCNs). The endpoints and bifurcation points of veins are chosen as nodes, and the vein vessels between nodes are edges. In order to overcome the over-smoothing problem and enhance the feature differentiation performance, a multi-hop graph mechanism is introduced to construct a multi-branch GCN. Meanwhile, an adaptive feature fusion module (AFFM) is introduced to adaptively fuse the node embeddings learned from multiple branches. An Arcface multi-classifier is adopted for the final recognition. Experimental results show that the proposed arm vein recognition method outperforms other matching recognition techniques and effectively handles the small sample problem. The ablation studies further confirm the effectiveness of the multi-hop graphs and the Arcface multi-classifier.