Adapting Large Language Models for Customer Service: A Comparative Study of RAG, RAFT and DoRA
摘要
Large language models (LLMs) are being increasingly deployed in customer-service chatbot applications. While traditional fine-tuning approaches often suffer from hallucinations or require computationally expensive retraining, emerging retrieval-based and parameter-efficient methods are increasingly regarded as promising alternatives. However, comprehensive evaluation of these paradigms in a customer-service context remains limited. To address this gap, we conduct a comprehensive comparison of three fine-tuning paradigms - Retrieval-Augmented Generation (RAG), Retrieval-Augmented Fine-Tuning (RAFT), and Weight-Decomposed Low-Rank Adaptation (DoRA) - for customer-service chatbot applications. All methods share a common backbone and are evaluated on the Bitext customer-support dataset. RAG method achieves strong factual consistency at the cost of higher inference latency. RAFT delivers the best overall balance of intent-classification accuracy, coverage, and low hallucination with moderate latency overhead. DoRA extends LoRA (Low Rank Adaptation) by decomposing weight updates into magnitude and directional components for precise low-rank adaptation. On Bitext, DoRA’s end-to-end fine-tuning not only achieves medium accuracy and coverage but also suffers a higher hallucination rate due to limited training data. In larger-scale settings (e.g., SQuAD + 400K-row FAQ), DoRA demonstrates surprising low hallucination and high BLEU/ROUGE, proving that dataset size significantly influences its performance. The results highlight trade-offs between latency, accuracy, and factual reliability, while emphasizing the importance of data scale and retrieval grounding in deploying scalable, trustworthy LLM-based customer-service systems.