Learning an effective global model on private and decentralized data sets has become an increasingly important challenge of machine learning in practice. Existing federated learning paradigms enable this via model aggregation that enforces a strong form of modeling homogeneity and synchronicity across clients. This is however not suitable to many practical scenarios. For example, in distributed sensing, heterogeneous sensors reading data from different views of the same phenomenon would need to use different models for different data modalities. Local learning therefore happens in isolation, but inference requires merging the local models to achieve consensus. This can be addressed with a feature fusion scheme that extracts local representations from local models and incorporates them into a global representation that improves the prediction performance. Achieving this in turn requires addressing two problems: (1) learning how to align similar feature components, which are arbitrarily arranged across clients, to enable representation aggregation and (2) constructing a consensus graph that captures the high-order interactions between local feature spaces and how to combine them to achieve a better prediction. This chapter will discuss recently developed solutions to these problems and illustrate their real-world applications in domains such as power grids and traffic networks, which are represented with time series data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Federated Learning of Models Pretrained on Different Features with Consensus Graphs

  • Tengfei Ma,
  • Jie Chen,
  • Trong Nghia Hoang

摘要

Learning an effective global model on private and decentralized data sets has become an increasingly important challenge of machine learning in practice. Existing federated learning paradigms enable this via model aggregation that enforces a strong form of modeling homogeneity and synchronicity across clients. This is however not suitable to many practical scenarios. For example, in distributed sensing, heterogeneous sensors reading data from different views of the same phenomenon would need to use different models for different data modalities. Local learning therefore happens in isolation, but inference requires merging the local models to achieve consensus. This can be addressed with a feature fusion scheme that extracts local representations from local models and incorporates them into a global representation that improves the prediction performance. Achieving this in turn requires addressing two problems: (1) learning how to align similar feature components, which are arbitrarily arranged across clients, to enable representation aggregation and (2) constructing a consensus graph that captures the high-order interactions between local feature spaces and how to combine them to achieve a better prediction. This chapter will discuss recently developed solutions to these problems and illustrate their real-world applications in domains such as power grids and traffic networks, which are represented with time series data.