Scholarlink

<p>The increasing intricacy and scale of deep learning (DL) models have heightened the necessity for effective weight and parameter optimization algorithms that sustain superior performance while reducing computational resource consumption. This paper thoroughly analyzes the evolution of parameter optimization strategies, ranging from initial methodologies to modern advancements, elucidating their principles and applications in natural language processing (NLP) and machine learning (ML). We pay special emphasis to parameter-efficient fine-tuning (PEFT) approaches, such as low-rank adaptation (LoRA) and its extensions, which facilitate the adaptation of large language models (LLMs) on resource-limited devices. These methods address challenges such as high computational demands, energy consumption, and deployment restrictions, thereby promoting more accessible and environmentally sustainable artificial intelligence (AI) solutions. By integrating methodological understandings with recent advancements, this survey underscores the essential role of parameter optimization in enabling scalable deep learning systems. It serves as an essential guide for researchers seeking to apply these techniques across diverse domains, emphasizing their impact on achieving efficient and robust model performance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

PEFT unlocked: methodologies, formulas, and applications in efficient LLM adaptation with a specific focus on LoRA

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月3日

查看全文
  • Maryam Najafi,
  • Ehsan Tavan,
  • Stefan Kuhn,
  • Salaheddin Alakkari,
  • Simon Colreavy

摘要

The increasing intricacy and scale of deep learning (DL) models have heightened the necessity for effective weight and parameter optimization algorithms that sustain superior performance while reducing computational resource consumption. This paper thoroughly analyzes the evolution of parameter optimization strategies, ranging from initial methodologies to modern advancements, elucidating their principles and applications in natural language processing (NLP) and machine learning (ML). We pay special emphasis to parameter-efficient fine-tuning (PEFT) approaches, such as low-rank adaptation (LoRA) and its extensions, which facilitate the adaptation of large language models (LLMs) on resource-limited devices. These methods address challenges such as high computational demands, energy consumption, and deployment restrictions, thereby promoting more accessible and environmentally sustainable artificial intelligence (AI) solutions. By integrating methodological understandings with recent advancements, this survey underscores the essential role of parameter optimization in enabling scalable deep learning systems. It serves as an essential guide for researchers seeking to apply these techniques across diverse domains, emphasizing their impact on achieving efficient and robust model performance.