DrIM: Context-Driven Nearest Neighbor Imputation Using Language Representation

Missing data poses significant challenges for machine learning and deep learning algorithms, which require complete datasets for training. In this paper, we aim to enhance post-imputation performance, measured by imputation utility. We introduce a k-nearest neighbors-based imputation method, DrIM, designed for heterogeneous (mixed-type) tabular datasets. DrIM leverages the representation learning capabilities of language models by transforming the tabular dataset into a text format and replacing the missing entries with [MASK] (or [UNK]) tokens. DrIM incorporates a contrastive learning framework and refines the representations. Moreover, our proposed method is theoretically justified by showing that contrastive learning induces a metric of representation space via density ratio estimation, thereby supporting its use for missing data imputation. To validate our proposed model, we evaluate its performance on missing data imputation across 10 real-world tabular datasets, demonstrating its ability to produce complete datasets with high imputation utility under various missing data scenarios.