Nowadays, many researchers have made significant progress in the field of multi-view 3D human pose estimation. However, numerous multi-view human pose estimation models based on deep learning heavily rely on data-driven training. As a 3D reconstruction method based on mathematical modeling, triangulation has shown excellent generalization ability and is widely used for 3D pose estimation and 3D pose annotation tasks in unlabeled environments. In this paper, we propose a refinement module based on graph convolution and visual fusion, and based on this, propose a triangulation-based method infused with structural information, Skeletal Algebraic Triangulation (SAT), encoding human pose prior knowledge into the model to ensure its robustness under occlusion and complex motions. Experiments show that our model outperforms algebraic methods and achieves comparable performance to state-of-the-art methods. Meanwhile, our method has better generalization performance, showing better and more robust results on different view Settings from the training dataset. Besides that, the proposed method can be applied to different backbone networks. As a core part of SAT, the graph refinement model can also be used to improve existing keypoint estimation. The volume triangulation combined with the graph refinement module, called Skeleton Volume triangulation, achieves state-of-the-art performance on the Human36M. In addition, the graph refinement module is also used for other keypoint estimation tasks that contain structural information, such as hand landmarks.