We focus on speeding up MobileViT, a model known for high accuracy in image classification tasks with fewer parameters. Our goal is to reduce the processing time of the Transformer part of MobileViT, which is the main slowdown factor, by using the Token Merging (ToMe) technique. However, because the number of tokens varies before and after merging, we cannot apply ToMe to MobileViT without making some modification. Therefore, we propose a simple method that facilitates the application of ToMe. By employing this method, we have successfully applied ToMe to MobileViT-s, leading to a slight drop in top-1 accuracy by 3.44% but reducing computational time by about 14.5%. While slightly inferior to the MobileViT-xs (smaller model), our model achieves similar speed and accuracy. Our method is simple and leaves the Token Merging algorithm unchanged. Thus, by improving the algorithm in addition to our proposed method, there is a potential to create models in the future that operate faster and with higher accuracy than MobileViT-xs. Additionally, since this modification can be adapted to other models facing similar issues, our proposal serves as a method that expands the adaptability of ToMe.