Scholarlink

<p>The heterogeneity of data between different clients limits the performance of federated learning training, and hyper-knowledge distillation has emerged as a potential solution to address heterogeneous data. However, existing hyper-knowledge distillation methods often sacrifice efficiency to improve model performance. To address these challenges, this paper proposes FedHDE, a novel federated learning framework based on hyper-knowledge distillation and efficient tensor optimization. FedHDE integrates a Dynamic Client Selection strategy within a teacher–assistant–student hierarchical structure to enhance global aggregation accuracy and reliability without relying on public datasets or server-side models. Furthermore, an efficient tensor optimization algorithm is introduced to improve numerical stability and reduce computation time. Theoretical analysis demonstrates that FedHDE achieves a convergence rate of <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(O(1/\sqrt{T})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>O</mi> <mo stretchy="false">(</mo> <mn>1</mn> <mo stretchy="false">/</mo> <msqrt> <mi>T</mi> </msqrt> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation>, ensuring efficient and stable learning. Experimental results on the CIFAR10 dataset show that FedHDE improves both local and global accuracy by 0.14%–4.32%, while reducing training time by 65.28% compared to state-of-the-art baselines.Code is public at <a href="https://anonymous.4open.science/r/FedHDE">https://anonymous.4open.science/r/FedHDE</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FedHDE: an efficient federated learning model based on hyper-knowledge distillation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月13日

查看全文
  • Xingang Zhang,
  • Chuang Liu,
  • Xiaofan Shao,
  • Tingzhi Li,
  • He Li,
  • Dongyan Zhang

摘要

The heterogeneity of data between different clients limits the performance of federated learning training, and hyper-knowledge distillation has emerged as a potential solution to address heterogeneous data. However, existing hyper-knowledge distillation methods often sacrifice efficiency to improve model performance. To address these challenges, this paper proposes FedHDE, a novel federated learning framework based on hyper-knowledge distillation and efficient tensor optimization. FedHDE integrates a Dynamic Client Selection strategy within a teacher–assistant–student hierarchical structure to enhance global aggregation accuracy and reliability without relying on public datasets or server-side models. Furthermore, an efficient tensor optimization algorithm is introduced to improve numerical stability and reduce computation time. Theoretical analysis demonstrates that FedHDE achieves a convergence rate of \(O(1/\sqrt{T})\) O ( 1 / T ) , ensuring efficient and stable learning. Experimental results on the CIFAR10 dataset show that FedHDE improves both local and global accuracy by 0.14%–4.32%, while reducing training time by 65.28% compared to state-of-the-art baselines.Code is public at https://anonymous.4open.science/r/FedHDE.