Scholarlink

A novel scalable approach to domain-specific enhancement of big language models is introduced in this work and demonstrated using LLaMA 3 for the fitness domain. Because they have not been exposed to specialized knowledge during pre-training, conventional large-scale language models often struggle with domain-specific reasoning. To address this, the proposed framework introduces two key innovations: (i) Dynamic Knowledge Alignment (DKA), which synergistically integrates general-purpose and domain-specific knowledge representations through a combination of knowledge distillation, contrastive learning, and adaptive weighting; and (ii) a multi-tiered distributed training architecture that leverages Fully Sharded Data Parallelism (FSDP), Model Parallelism, and ZeRO-3 optimization for efficient fine-tuning of the 70B parameter LLaMA 3 model. Experiments on both classification and regression tasks, utilizing real-world fitness datasets, demonstrate substantial improvements across accuracy, F1-score, and regression error metrics. The proposed model consistently outperforms baseline models, including GPT-2, BERT, and unadapted LLaMA 3, achieving up to 94.8% classification accuracy and 0.94 R^2 regression performance.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Scalable Structure for Expert Module-Based Domain-Specific Adaptation of Large Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Abha Kiran Rajpoot,
  • Gaurav Agrawal,
  • Diksha Dani,
  • Jagendra Singh,
  • Neha Yadav

摘要

A novel scalable approach to domain-specific enhancement of big language models is introduced in this work and demonstrated using LLaMA 3 for the fitness domain. Because they have not been exposed to specialized knowledge during pre-training, conventional large-scale language models often struggle with domain-specific reasoning. To address this, the proposed framework introduces two key innovations: (i) Dynamic Knowledge Alignment (DKA), which synergistically integrates general-purpose and domain-specific knowledge representations through a combination of knowledge distillation, contrastive learning, and adaptive weighting; and (ii) a multi-tiered distributed training architecture that leverages Fully Sharded Data Parallelism (FSDP), Model Parallelism, and ZeRO-3 optimization for efficient fine-tuning of the 70B parameter LLaMA 3 model. Experiments on both classification and regression tasks, utilizing real-world fitness datasets, demonstrate substantial improvements across accuracy, F1-score, and regression error metrics. The proposed model consistently outperforms baseline models, including GPT-2, BERT, and unadapted LLaMA 3, achieving up to 94.8% classification accuracy and 0.94 R^2 regression performance.