Unsupervised interactive segmentation aims to partition images based on user-provided interaction cues without relying on pre-prepared annotation supervision. Existing methods suffer from systemic biases caused by inherent flaws in unsupervised pseudo-label generation, such as spatial inaccuracies and semantic inconsistencies. These issues propagate into interaction simulation, creating referent misalignment between the simulated intent and pseudo-label, thereby impairing the model’s ability to capture authentic interaction logic. To address this problem, we propose a referent-aligned framework that redefines interaction simulation by prioritizing user intent through a “click-to-mask” paradigm. Our methodology addresses spatial and semantic misalignment via semantic-guided probabilistic sampling and click-controlled segmentation, ensuring alignment between user intent and segmentation targets. Experimental results demonstrate that our method achieves significant performance improvements in unsupervised interactive segmentation task.