Behind the booming decentralized finance (DeFi) ecosystem driven by blockchain technology, various financial risks lurking, including money laundering, gambling, Ponzi schemes, and phishing. Due to the decentralization and anonymity of Ethereum, Ponzi schemes can be easily deployed, causing huge economic losses to investors. Existing detection methods based on transaction data are difficult to provide early risk warnings, while detection methods based on smart contract source code and opcodes have insufficient feature fusion at multiple levels. We propose an approach for constructing a multi-level Heterogeneous Semantic Graph (HSG) of smart contracts, and improve the HAN model to detect Ponzi scheme smart contracts based on the Heterogeneous Semantic Graph. The experimental results demonstrate the effectiveness of our approach, achieving an accuracy of 97.21%, a precision of 94.29%, and an F1 score of 90.41%.