With the rapid development of blockchain technology, vulnerabilities in smart contracts have become a major threat to asset security. Traditional rule-based detection methods, although interpretable, often suffer from high false positive rates and limited scalability. Despite recent progress, deep learning methods are often limited to unimodal approaches and lack the capability for fine-grained analysis. Our analysis of 14 common vulnerabilities revealed that function-level granularity strikes the optimal balance between detection accuracy and efficiency. Building on this observation, we propose the Jakiro method, which improves detection accuracy by integrating the semantic information from control flow graphs (CFGs) and source code using cross-modal contrastive learning. Experiments conducted on a dataset of more than 38,000 real-world contracts demonstrate that Jakiro surpasses the majority of the 10 baseline methods across three tasks: reentry, integer overflow, and transaction order dependency, achieving average improvements of 6.49%, 2.66%, and 4.62% in precision, recall, and F1 score, respectively.