Existing unsupervised salient object detection (USOD) methods typically rely on pseudo labels generated by traditional algorithms or class activation maps (CAM). However, these labels are often noisy and incomplete, which limits the performance of the final model. To tackle this issue, we propose a novel two-stage USOD framework. In the first stage, we focus on generating high-quality pseudo labels. Specifically, we first utilize a ViT model pretrained on DINO to extract features using a contrastive and binarization loss, producing initial coarse masks. These coarse masks are then refined using the unsupervised Segment Anything Model (UnSAM) with multi-point prompting strategy. More importantly, we introduce an iterative label noise removal mechanism based on cross-validation, which identifies and corrects noisy pseudo labels in a self-supervised manner without any annotations. In the second stage, we propose one novel salient object detection (SOD) model, the purified high-quality pseudo labels are used as supervision for training the SOD network. Extensive experiments on public datasets demonstrate that our method outperforms existing unsupervised approaches and achieves state-of-the-art performance.