Knowledge distillation, traditionally used to train compact student networks through deep teacher models, has been successfully applied to many vision tasks. While Convolutional Neural Networks (CNNs) have been extensively studied to enhance the performance of smaller models, this approach remains unexplored for Vision Transformers (ViTs) in camera relocalization tasks. ViTs offer unique architectural benefits for advancing vision-based understanding, but they are challenging to deploy. In this paper, we introduce LocKD, a novel method optimized for integrating compact ViT models into camera relocalization tasks. LocKD innovatively leverages the intrinsic features within ViTs and introduces dual modules: a feature mimicking module for the nuanced capture of shallow layer details, which are crucial for constructing attention maps, and an attention generation module aimed at the deeper layers to ensure a richer assimilation of semantic content. Furthermore, LocKD synergistically combines with logit-based distillation methods to enhance the performance of the student model. Specifically, empirical validations on the Oxford RobotCar dataset demonstrate that our approach yields a 7.14% and 11.1% improvement compared to EffLoc-Small. This shows the potential of our method in improving the efficiency and effectiveness of camera relocalization tasks using compact ViT models.