Grokking is a phenomenon where a model, after initially overfitting, eventually achieves near‐perfect validation performance after an extended period of training. In this paper, our aim is to understand how rigid the learned representations of a grokked model are. We define rigidity in terms of a model’s inability to adapt its internal representations when faced with a new task. To examine this, we pre-trained a multi-layer perceptron (MLP) on MNIST using three strategies: standard training on the full dataset, standard training on a reduced subset, and grokking-induced training with large weight initialization on the subset. Then we fine-tuned each model on Fashion‑MNIST. Although the grokked model reached 100% training accuracy and 90.36% validation accuracy on MNIST, its performance on Fashion‑MNIST dropped drastically to around 16.5%. Analysis of the loss landscape and Hessian eigenvalues reveals that grokking drives the model into a deep minimum with a high density of near-zero eigenvalues, indicating an overly rigid parameter space and over-parametrization. Our results demonstrate that while grokking can yield better performance, it produces specialized and inflexible representations that severely impact adaptation.