Scholarlink

The rapid evolution of Large Language Models (LLMs) has revolutionized natural language processing; however, their deployment in heterogeneous environments remains challenging. While Parameter-Efficient Fine-Tuning (PEFT) has emerged as a cost-effective adaptation strategy, existing federated learning approaches fail to effectively address the combined challenges of data heterogeneity, computational diversity, and privacy preservation. This paper introduces DynamicFedPEFT, a novel federated learning framework that dynamically optimizes LLM adaptation through three key innovations: (1) a multi-dimensional evaluation framework that quantifies client data quality using semantic coherence, lexical diversity, and contextual richness; (2) an adaptive LoRA configuration mechanism that automatically adjusts rank and scaling parameters based on local data characteristics; and (3) a quality-weighted aggregation protocol that prioritizes contributions from high-value clients. Furthermore, the framework incorporates a resource-aware training architecture that enables full participation across heterogeneous devices through progressive parameter freezing. Comprehensive evaluations on six NLP benchmarks demonstrate state-of-the-art performance, with a 5.2% accuracy improvement over conventional federated PEFT approaches and a 28% reduction in communication costs. The proposed solution establishes a new paradigm for collaborative LLM optimization, balancing model performance, resource efficiency, and data privacy.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DynamicFedPEFT: Efficient Fine-Tuning of Dynamic Federated Parameters for Large Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Xiaorui Luo,
  • Chi Jiang,
  • Shuai Wang,
  • Yin Zhang

摘要

The rapid evolution of Large Language Models (LLMs) has revolutionized natural language processing; however, their deployment in heterogeneous environments remains challenging. While Parameter-Efficient Fine-Tuning (PEFT) has emerged as a cost-effective adaptation strategy, existing federated learning approaches fail to effectively address the combined challenges of data heterogeneity, computational diversity, and privacy preservation. This paper introduces DynamicFedPEFT, a novel federated learning framework that dynamically optimizes LLM adaptation through three key innovations: (1) a multi-dimensional evaluation framework that quantifies client data quality using semantic coherence, lexical diversity, and contextual richness; (2) an adaptive LoRA configuration mechanism that automatically adjusts rank and scaling parameters based on local data characteristics; and (3) a quality-weighted aggregation protocol that prioritizes contributions from high-value clients. Furthermore, the framework incorporates a resource-aware training architecture that enables full participation across heterogeneous devices through progressive parameter freezing. Comprehensive evaluations on six NLP benchmarks demonstrate state-of-the-art performance, with a 5.2% accuracy improvement over conventional federated PEFT approaches and a 28% reduction in communication costs. The proposed solution establishes a new paradigm for collaborative LLM optimization, balancing model performance, resource efficiency, and data privacy.