Scholarlink

<p>Integrating machine learning into customer service chatbots has substantially improved their ability to interpret and respond to user queries. However, existing evaluation practices often treat such systems as black boxes or assess individual modules in isolation, limiting insight into the architectural sources of performance variation. This study proposes a component-complete evaluation framework for goal-oriented customer service chatbots implemented using a modular pipeline architecture. The framework enables systematic, component-wise benchmarking of Natural Language Understanding, Dialogue Management, and Natural Language Generation under controlled and standardized experimental conditions. Using representative neural architectures for each pipeline component, we conduct a structured evaluation on the MultiWOZ dataset. Component performance is assessed using task-appropriate functional metrics and statistical validation to reduce inconsistencies arising from optimization variability. The results reveal clear trade-offs across components, demonstrating differing strengths of contextualized language models, recurrent architectures, and value-based reinforcement learning methods within modular dialogue systems. Beyond individual model comparisons, the primary contribution of this work is a structured evaluation methodology that supports fine-grained architectural diagnosis across the full chatbot pipeline under controlled experimental settings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Experimental evaluation of machine learning models for goal-oriented customer service chatbot with pipeline architecture

  • 期刊论文

  • 发表日期：

    2026年8月3日

查看全文
  • Ezzatul Akmal Kamaru Zaman,
  • Nurul Ain Nabilah Mohd Isa,
  • Siti Nuraishah Agos Jawaddi,
  • Azlan Ismail

摘要

Integrating machine learning into customer service chatbots has substantially improved their ability to interpret and respond to user queries. However, existing evaluation practices often treat such systems as black boxes or assess individual modules in isolation, limiting insight into the architectural sources of performance variation. This study proposes a component-complete evaluation framework for goal-oriented customer service chatbots implemented using a modular pipeline architecture. The framework enables systematic, component-wise benchmarking of Natural Language Understanding, Dialogue Management, and Natural Language Generation under controlled and standardized experimental conditions. Using representative neural architectures for each pipeline component, we conduct a structured evaluation on the MultiWOZ dataset. Component performance is assessed using task-appropriate functional metrics and statistical validation to reduce inconsistencies arising from optimization variability. The results reveal clear trade-offs across components, demonstrating differing strengths of contextualized language models, recurrent architectures, and value-based reinforcement learning methods within modular dialogue systems. Beyond individual model comparisons, the primary contribution of this work is a structured evaluation methodology that supports fine-grained architectural diagnosis across the full chatbot pipeline under controlled experimental settings.