Vision Transformers (ViTs) have achieved the excellent global modeling capability due to self-attention mechanism. Contrary to convolution operations, recent studies suggest that self-attentions behave like low-pass filters and enhancing their high-pass filtering capability can improve the performance of ViT-based models. While several attempts have been made to promote the high-frequency characteristics of ViTs, they have all faced the issue on how to find more generalized features. To handle this, we present a CurveViT that adaptively reconfigures the low- and high-frequency features. Especially, the spatial features are transformed into frequency domain to adaptively control the generation of a group of parameterized curves. We utilize the frequency-aware curve tokens as an effective complementation of multi-head self-attention to balance multi-band frequency components, which further enhance the representation ability of ViTs. Extensive experiments demonstrate the effectiveness of the proposed CurveViT and show that our method achieves significant improvements compared to the baseline models in typical vision tasks such as image classification, object detection, and semantic segmentation.