Parkinson’s Disease (PD) is a typical neurodegenerative disease, and early diagnosis is crucial to delaying the progression of the disease. However, existing PD diagnosis methods using sMRI and fMRI fail to design modality-specific feature extraction networks tailored to the unique characteristics of each modality. In addition, most of these methods do not deeply explore the feature correlation of different modalities in the feature fusion module. To tackle these challenges, we propose a Spatial-temporal Dual-pathway Network with Multi-scale Feature Fusion (SDMFF) framework to enhance the performance of PD diagnosis. Specifically, we develop a spatial-temporal dual-pathway network capable of effectively extracting representations from sMRI and fMRI. For sMRI, we design a spatial CNN-transformer module to extract both local and global structural features. For fMRI, we design a spatial-temporal Transformer to capture dynamic spatial-temporal features. To effectively fuse the information from both sMRI and fMRI, we design a multi-scale convolutional attention feature fusion module, which fully integrates multi-scale feature information of sMRI and fMRI. Extensive experimental results demonstrate that our proposed SDMFF achieves state-of-the-art performance on both public and private datasets, with accuracy of 0.926 and 0.858, respectively.