A new transformer-based generative chatbot using CycleGAN approach
摘要
Chatbots are computer programs that interact with humans in a natural language format. To create more sophisticated chatbots, developers often combine natural language processing (NLP) with sequential models to generate phrases. One limitation of these methods is their sequential training, that can lead to inaccuracies. Traditional Seq2Seq models further encounter challenges such as dependency on diverse, high-quality training data, a propensity to generate generic responses, and error propagation during inference. These issues collectively undermine the accuracy, coherence, and overall quality of the generated phrases. To address this limitation, a new generative architecture that leverages Cycle Generative Adversarial Networks (CycleGAN) and transformer models is proposed. The main novelty of this approach is semantically improving generated conversations by learning a mapping between human and bot phrases. This integration leverages the complementary strengths of both frameworks. CycleGAN ensures semantic consistency through cycle-consistency and identity losses, while adversarial losses enhancing generalizability. Given the transformer model’s ability to process long sequences in parallel, proposed model can effectively enhance the quality of generated phrases in human-bot conversations. The proposed architecture includes two generators and two discriminators. Both generators utilize the full transformer model to generate fake phrases, while the discriminators employ only the encoder part of a transformer model followed by a classifier. Extensive experiments on the Persona-Chat, Daily Dialogue and DSTC7 datasets consistently demonstrated the superiority of proposed approach over state-of-the-art methods. We achieved average improvements of about 2.12, 1.51 and 0.12 in BLEU, ROUGE, and Word embedding-based metrics, respectively.