Synthetic data is commonly used today in machine learning. From the traditional statistical approach of imputing missing information, the data generation techniques have now leaned towards machine learning and artificially intelligent generative techniques. Generative Adversarial Networks (GAN) and Variational Autoencoder (VAE) are the techniques employed. Tabular data is most common in industrial research. TVAE generates synthetic data which is tabular and available as a part of synthetic data vault (SDV) introduced by Neha Patki et al. Hence, we pick TVAE, perform an in-depth analysis on existing literature and experimentally highlight the strengths and limitations of TVAE, conducted using different types of tabular data. TVAE as such, completely ignores the minority samples in the dataset. Hence, we modify the data generation algorithm by introducing another set of encoder and decoder to oversample only those categorical sub classes which is small in representation and provide that information to the latent space. Our experiments reveal that, despite the several techniques proposed in various literature regarding sampling and balancing the data using VAE, TVAE struggles in producing the categorical values for a subcategory and the associated target class within a feature when the sub-categorical value is a minority. Based on our experiments and analysis, we see a need for further study on modeling the TVAE to produce a well-balanced dataset.