Scholarlink

<p>Interactive chatbots powered by large language models (LLMs) are increasingly utilized for disseminating organizational information. However, deploying them in closed domains presents challenges such as data sparsity, domain-specific biases, and context maintenance, which can lead to model hallucination. To tackle these challenges, we propose a generative hybrid LLM-based architecture inspired by few-shot learning. This approach consists of three key stages: (1) developing a similarity model to identify similar conversations, (2) generating optimized prompts to enhance response relevance, and (3) fine-tuning the LLM using the low-rank adaptation (LoRA) method with a custom dataset. Given the critical role of prompts in guiding LLM outputs, proposed approach dynamically retrieves similar questions and incorporates them into the input prompt, enabling the fine-tuned model to generate semantically precise responses. This strategy significantly improves response quality, particularly in domain-specific inquiries. Extensive experiments on benchmark datasets—including PersonaChat, DailyDialog, and DSTC7—demonstrate the superiority of proposed approach over existing state-of-the-art methods, yielding average improvements of 0.58, 0.73, and 1.23 in BLEU, ROUGE, and word embedding-based evaluation metrics, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A few-shot learning approach with similarity-based prompt engineering for fine-tuning LLMs in closed-domain conversational AI

  • 期刊论文

  • 发表日期：

    2026年4月21日

查看全文
  • Nura Esfandiari,
  • Kourosh Kiani,
  • Razieh Rastgoo

摘要

Interactive chatbots powered by large language models (LLMs) are increasingly utilized for disseminating organizational information. However, deploying them in closed domains presents challenges such as data sparsity, domain-specific biases, and context maintenance, which can lead to model hallucination. To tackle these challenges, we propose a generative hybrid LLM-based architecture inspired by few-shot learning. This approach consists of three key stages: (1) developing a similarity model to identify similar conversations, (2) generating optimized prompts to enhance response relevance, and (3) fine-tuning the LLM using the low-rank adaptation (LoRA) method with a custom dataset. Given the critical role of prompts in guiding LLM outputs, proposed approach dynamically retrieves similar questions and incorporates them into the input prompt, enabling the fine-tuned model to generate semantically precise responses. This strategy significantly improves response quality, particularly in domain-specific inquiries. Extensive experiments on benchmark datasets—including PersonaChat, DailyDialog, and DSTC7—demonstrate the superiority of proposed approach over existing state-of-the-art methods, yielding average improvements of 0.58, 0.73, and 1.23 in BLEU, ROUGE, and word embedding-based evaluation metrics, respectively.