<p>Chatbots are computer programs that interact with humans in a natural language format. To create more sophisticated chatbots, developers often combine natural language processing (NLP) with sequential models to generate phrases. One limitation of these methods is their sequential training, that can lead to inaccuracies. Traditional Seq2Seq models further encounter challenges such as dependency on diverse, high-quality training data, a propensity to generate generic responses, and error propagation during inference. These issues collectively undermine the accuracy, coherence, and overall quality of the generated phrases. To address this limitation, a new generative architecture that leverages Cycle Generative Adversarial Networks (CycleGAN) and transformer models is proposed. The main novelty of this approach is semantically improving generated conversations by learning a mapping between human and bot phrases. This integration leverages the complementary strengths of both frameworks. CycleGAN ensures semantic consistency through cycle-consistency and identity losses, while adversarial losses enhancing generalizability. Given the transformer model’s ability to process long sequences in parallel, proposed model can effectively enhance the quality of generated phrases in human-bot conversations. The proposed architecture includes two generators and two discriminators. Both generators utilize the full transformer model to generate fake phrases, while the discriminators employ only the encoder part of a transformer model followed by a classifier. Extensive experiments on the Persona-Chat, Daily Dialogue and DSTC7 datasets consistently demonstrated the superiority of proposed approach over state-of-the-art methods. We achieved average improvements of about 2.12, 1.51 and 0.12 in BLEU, ROUGE, and Word embedding-based metrics, respectively.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A new transformer-based generative chatbot using CycleGAN approach

  • Nura Esfandiari,
  • Kourosh Kiani,
  • Razieh Rastgoo

摘要

Chatbots are computer programs that interact with humans in a natural language format. To create more sophisticated chatbots, developers often combine natural language processing (NLP) with sequential models to generate phrases. One limitation of these methods is their sequential training, that can lead to inaccuracies. Traditional Seq2Seq models further encounter challenges such as dependency on diverse, high-quality training data, a propensity to generate generic responses, and error propagation during inference. These issues collectively undermine the accuracy, coherence, and overall quality of the generated phrases. To address this limitation, a new generative architecture that leverages Cycle Generative Adversarial Networks (CycleGAN) and transformer models is proposed. The main novelty of this approach is semantically improving generated conversations by learning a mapping between human and bot phrases. This integration leverages the complementary strengths of both frameworks. CycleGAN ensures semantic consistency through cycle-consistency and identity losses, while adversarial losses enhancing generalizability. Given the transformer model’s ability to process long sequences in parallel, proposed model can effectively enhance the quality of generated phrases in human-bot conversations. The proposed architecture includes two generators and two discriminators. Both generators utilize the full transformer model to generate fake phrases, while the discriminators employ only the encoder part of a transformer model followed by a classifier. Extensive experiments on the Persona-Chat, Daily Dialogue and DSTC7 datasets consistently demonstrated the superiority of proposed approach over state-of-the-art methods. We achieved average improvements of about 2.12, 1.51 and 0.12 in BLEU, ROUGE, and Word embedding-based metrics, respectively.