Action Quality Assessment (AQA) has emerged as a burgeoning topic due to its wide applicability across various domains. However, most existing approaches score action videos by focusing on the deep features of the entire video, which lacks a fine-grained understanding of human motions, thereby limiting the accuracy and interpretability of AQA. In this paper, we consider both video features and human poses to assess action quality accurately. Human poses can assist in locating the concerned spatial position in image frames, thereby providing a precise execution of specific actions. To achieve this goal, we propose an effective dual-stream spatial position-aware AQA framework that integrates both video and human pose features to evaluate the actions. Specifically, we design a spatial position-aware (SPA) module to enhance the representation of local features at the spatial position of the athlete. Extensive experiments demonstrate that our approach achieves good performances on common AQA datasets, validating the effectiveness of the proposed method.