RAP: Random Projection is What You Need for Vertical Federated Learning

Vertical federated learning (VFL) considers model training when the features of data samples are partitioned over a set of clients. As the standard practice of VFL, SplitNN decomposes a model into a bottom part on the clients and a top part on a server, and requires the clients and server to exchange activations/gradients in every mini-batch. We observe that SplitNN is inefficient due to frequent client-server communication and propose random projection (i.e., RAP) to improve efficiency. RAP is radically simple, i.e., the clients transform their local features and transfer the transformed data to the server, and the server trains on the transformed data without communicating with the clients. As only one round of client-server communication is required, RAP is much more efficient than SplitNN. RAP uses Gaussian projection matrix as the data transformation. Data privacy is preserved because the projection matrix of each client is private and resembles the bottom model of SplitNN. Model accuracy is not affected because the Gaussian projection matrix is invertible and preserves the geometry of the original space. To evaluate RAP, we experimented with both regression and classification tasks on six datasets. The results show that RAP matches SplitNN in model accuracy and accelerates training by over 630 \(\times \) .