The quality of data is of paramount importance when training a Machine Learning model, especially in sensitive domains. Since model performance is directly dependent on data quality, it is critical to ensure that high-quality data samples are used during training. In Federated Learning, however, the inability to assess data directly limits our understanding of its quality, which can undermine confidence in the model’s reliability. This issue is of particular importance in domains, such as those dealing with medical records, where data integrity is essential. To address this challenge, we propose a novel approach to quality assurance in Federated Learning, called FedQAM (Federated Quality Assurance Mechanism). Our method evaluates the performance of client models on a publicly available test set and compares their results against a predefined threshold at the client side before incorporating them into the global model. This ensures that only reliable client updates are aggregated. Additionally, we introduce and evaluate various threshold selection schemes to optimise the process. Our approach effectively identifies and filters out unreliable or malicious clients, preventing them from compromising the model’s integrity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FedQAM: Quality Assurance Mechanism in Federated Learning Using Client-Side Testing and Thresholding

  • Mashal Khan,
  • Matthias Nickles,
  • Frank G. Glavin

摘要

The quality of data is of paramount importance when training a Machine Learning model, especially in sensitive domains. Since model performance is directly dependent on data quality, it is critical to ensure that high-quality data samples are used during training. In Federated Learning, however, the inability to assess data directly limits our understanding of its quality, which can undermine confidence in the model’s reliability. This issue is of particular importance in domains, such as those dealing with medical records, where data integrity is essential. To address this challenge, we propose a novel approach to quality assurance in Federated Learning, called FedQAM (Federated Quality Assurance Mechanism). Our method evaluates the performance of client models on a publicly available test set and compares their results against a predefined threshold at the client side before incorporating them into the global model. This ensures that only reliable client updates are aggregated. Additionally, we introduce and evaluate various threshold selection schemes to optimise the process. Our approach effectively identifies and filters out unreliable or malicious clients, preventing them from compromising the model’s integrity.