Federated Gradient Boosting Decision Trees (GBDT) have gained popularity for enabling collaborative, privacy-preserving training across multiple distributed participants. However, existing federated GBDT frameworks require extensive communications for the creation of each subtree, as each participant trains data locally. These methods often involve complex secure multi-party computation or homomorphic encryption techniques that hinder training efficiency and suffer from low model accuracy in uneven data distributions. To address these issues, we propose a Masked Aggregation Learning (MAL) framework for federated GBDT. MAL combines distributed data preprocessing with centralized training, allowing participants to securely mask their data and share it with a central server for centralized training. Our approach includes constructing decision trees using histograms by discretizing continuous feature values into distinct buckets. During data preprocessing, we introduce a Secret Extremes Bucket Construction (SE-Bucket) method based on order-revealing encryption for unified bucketing and feature value obfuscation. Additionally, we propose an Erasable Label Mask generation algorithm that ensures label privacy while the masks do not impact model accuracy. MAL reduces the communication rounds from a linear relationship with the number of GBDT subtrees to a constant two rounds, independent of the number of subtrees, and maintains model accuracy regardless of how the data is distributed among the participants. Experimental results show that our method, MAL, achieves accuracy comparable to centralized training while being 500 to 600 times faster than state-of-the-art federated decision tree training solutions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Masked Aggregation Learning for Enhancing Distributed Gradient Boosting Decision Trees

  • Yuting Zha,
  • Chao Lin,
  • Xinyi Huang,
  • Dugang Liu

摘要

Federated Gradient Boosting Decision Trees (GBDT) have gained popularity for enabling collaborative, privacy-preserving training across multiple distributed participants. However, existing federated GBDT frameworks require extensive communications for the creation of each subtree, as each participant trains data locally. These methods often involve complex secure multi-party computation or homomorphic encryption techniques that hinder training efficiency and suffer from low model accuracy in uneven data distributions. To address these issues, we propose a Masked Aggregation Learning (MAL) framework for federated GBDT. MAL combines distributed data preprocessing with centralized training, allowing participants to securely mask their data and share it with a central server for centralized training. Our approach includes constructing decision trees using histograms by discretizing continuous feature values into distinct buckets. During data preprocessing, we introduce a Secret Extremes Bucket Construction (SE-Bucket) method based on order-revealing encryption for unified bucketing and feature value obfuscation. Additionally, we propose an Erasable Label Mask generation algorithm that ensures label privacy while the masks do not impact model accuracy. MAL reduces the communication rounds from a linear relationship with the number of GBDT subtrees to a constant two rounds, independent of the number of subtrees, and maintains model accuracy regardless of how the data is distributed among the participants. Experimental results show that our method, MAL, achieves accuracy comparable to centralized training while being 500 to 600 times faster than state-of-the-art federated decision tree training solutions.