A Meta Learning-Based Training Algorithm for Robust Dialogue Generation

There are many low-resource scenarios in the field of dialogue generation, such as medical diagnosis. Many dialogue generation models in these scenarios are usually unstable due to the lack of training corpus. As one of the most popular training algorithms in recent years, meta learning has achieved remarkable results. MAML in meta learning can find a fast adaptive initialization parameter for a series of low resource tasks, which is often used to solve the problem of low resource, and has achieved excellent performance in image classification tasks. However, in the field of text generation, such as dialogue generation, because of the large vocabulary, long sequence and large number of parameters involved in text generation, the effect of MAML is unstable. Therefore, this paper proposes a high robust text generation training framework based on meta learning for the dialogue generation task. By identifying the significant information in the model parameters, the optimizer can train the important parameters more concentrated in the limited data in the bi-level optimization. Experiments show that our method has a good performance on the BLEU scores on the six single-domain datasets.