Similarity Based on Resample Exposure
摘要
Measuring distance and similarity is one of the most fundamental problems in data science and statistics. Proximity functions are used almost everywhere, from clustering problems to outlier detection, database systems, and data privacy. One important challenge is how to handle heterogeneous data, which makes up a large part of real-world tabular data. Euclidean distance with dummy variables is only half a solution, and Gower’s similarity, which is usually the go-to choice for similarity estimation in heterogeneous data, also lacks stability when it comes to combinations of categorical variables. In this paper, we propose Resample Exposure, as a novel heterogeneous measure that effectively captures the similarity of records based on resampling probability and distance with traversal penalty. Resample exposure integrates well with various distance- or similarity-based tasks; we show that the measure is competitive for nearest neighbour classification and viable for partitioning around medoids, making resample exposure a relevant measure for heterogeneous data applications.