Accurate polyp segmentation is important for early colorectal cancer detection and effective treatment planning. However, existing methods struggle to precisely segment polyps with diverse morphologies, varying sizes, and ambiguous boundaries because they cannot simultaneously capture the contextual information in both the spatial and frequency domains. To address these limitations, we propose a novel segmentation network that synergistically combines multi-scale spatial context and learnable high-frequency domain augmentation. Specifically, our architecture introduces two major innovations: (1) a multi-scale dilated convolution module that efficiently aggregates multi-scale features through parallel dilated convolutions in the spatial domain, and (2) a high-frequency augmentation that selectively enhances high-frequency components such as edges and textures of polyps via learning FFT spectrum soft masks in the frequency domain. Importantly, these proposed two components are fully differentiable and easily pluggable, making them broadly applicable to different medical image segmentation backbones. Experimental results on five public polyp datasets demonstrate that our approach not only captures global structure effectively but also preserves fine-grained boundary details, outperforming state-of-the-art methods with consistent cross-dataset performance while maintaining superior computational efficiency.