Large-scale pre-training equips vision foundation models with impressive image-understanding ability, yet prompt-based frameworks such as SAM still depend on manually supplied cues. In few-shot semantic segmentation, the personalized variant PerSAM replaces these cues with positive and negative points sampled from a location-prior heatmap; however, this location-prior heatmap frequently blurs object boundaries and introduces background noise, hindering precise prompting. We therefore propose SageNet, a training-free framework that unifies geometric alignment, semantic discrimination, and prompt diversity in a single pipeline. SageNet first fuses complementary visual-similarity cues to robustly align support and query images and generate a coarse similarity map. It then converts CLIP-derived text–image cosine similarities into a semantics-aware guidance map and merges it with the visual prior, thereby enhancing foreground activation and suppressing background response. Finally, a farthest-point sampler draws spatially uniform positive and negative prompts from the refined heatmap, guiding a frozen SAM to produce coherent, semantically consistent masks without any parameter updates. By concurrently improving alignment, semantic guidance, and balanced prompting, SageNet remains highly robust to extreme viewpoint changes and other challenging conditions, delivering performance on DAVIS-2017, PerSeg and FSS-1000 that surpasses current state-of-the-art methods.