VF-FD: Feature Deduplication for Vertical Federated Learning

Vertical federated learning (VFL) assumes that the features of data samples are scattered over the clients. In practice, the clients may hold overlapping and noisy features, and we conduct feature deduplication by choosing one value from these overlapping features for each sample. Existing feature selection methods cannot handle feature deduplication because they choose the same set of features for all samples while feature deduplication is more flexible and can choose different features for different samples. As such, we propose a method called VF-FD to conduct feature deduplication. VF-FD first identifies groups of possibly overlapping features and then conducts selection in each overlapping group. In particular, VF-FD uses the earth mover distance (EMD) to quantify the similarity of features and decides that two features are overlapping when their EMD is small. For selection, VF-FD trains a surrogate model with all features and deduces a loss sensitive score (LSS) to quantify the influence of each feature on the loss. For each overlapping feature group, a sample selects the feature component that minimizes its loss. We experiment VF-FD on 6 datasets and compare it with state-of-the-art feature selection methods. The results show that VF-FD can improve model accuracy by more than 10% over the best baseline.