FT-BERT-GAN: an enhanced Feature-Type-Aware BERT (FT-BERT) with GAN for privacy-preserving synthetic text generation
摘要
In domains such as healthcare, finance, and customer service, tabular datasets often combine structured metadata with unstructured free-text descriptions. It is always a challenging task to preserve privacy and to provide high-quality data augmentation for mixed-format datasets in machine learning algorithms. We propose FT-BERT-GAN, a technique that combines a Feature-Type-Aware BERT (FT-BERT) with a conditional Generative Adversarial Network (GAN) to generate privacy-preserving synthetic tabular and text data. FT-BERT strengthens the basic BERT architecture by incorporating learnt embeddings for feature types alongside token and positional embeddings. It enables the model to obtain both semantic and structural contexts across heterogeneous data. These enhanced embeddings are then fed into a conditional GAN along with a differential privacy method to generate realistic samples while preserving robust privacy. The generated data retain the original dataset’s statistical and contextual reliability while avoiding duplication of sensitive records. The proposed method is evaluated on benchmark datasets such as MIMIC-III, UCI Adult, and Amazon Reviews. It demonstrates that the proposed FT-BERT-GAN method consistently outperforms existing methods CTAB-GAN + and DP-RVAE, by improving downstream classification in unbalanced scenarios and by overcoming Membership Inference Attacks (MIA). The results of this study demonstrate the FT-BERT-GAN method as a realistic and generalizable framework for privacy-preserving synthetic data generation, providing an effective, safe, and efficient method for data augmentation in sensitive, mixed-format domains.