Vision Transformers (ViTs) have shown remarkable potential in face recognition task. However, the vanilla ViTs suffer from an over-smoothing issue, that the local tokens progressively converge towards a uniform representation with the increasing depth. Such an over-smoothing issue impairs the diversity of patch representation, suppresses High-Frequency (HF) component, and thus leads to degenerated performance. To this end, in this paper, we incorporate a variational module into the ViT framework and thus propose a novel approach, called Variational ViT (V-ViT), for robust face recognition. Specifically, in V-ViT, the feature of each local patch is modeled as a Gaussian distribution, which can not only enable the uncertainty learning in patch level but also mitigate the over-smoothing issue inherently. Moreover, we decompose the sampled local tokens into Direct Current (DC) components and HF components, and amplify the HF components by dynamically inhibiting the proportion of DC components. We conduct extensive experiments on nine benchmark datasets and the experimental results demonstrate the effectiveness of our proposed V-ViT approach. The code is available at https://github.com/yuying9/V-ViT .