Pre-trained language models, like Bidirectional Encoder Representations from Transformers (BERT), have reached the highest level in many natural speech tasks. On the other hand, the size of the embedding matrices makes it complicated to tailor such models to specific domains or use them in resource-constrained environments. This research work present an innovative method that applies Variational Autoencoder (VAE) to compress token embeddings of a topic-specific vocabulary, thereby significantly reducing the model’s size while retaining its performance at the task. VAE-based compression is used on the medical corpus and show that a compressed model attains accuracy at the level of 89% and F1-score of 0.88, which is very close to the model. Furthermore, our approach includes memory constriction to 500 MB, and inference latency to 110 ms is implemented and the results prove that it works better than the traditional approaches such as frequency-based and TF-IDF-based pruning not only in terms of accuracy but also with resources I/O efficiency. This experiment introduces a positive proposal for the use of pre-trained language models in the sectors like healthcare and finance through real-time application.