KELTP: Keyword-Enhanced Learned Token Pruning for Knowledge-Grounded Dialogue

One of the important demands for open-domain conversational agents is to generate appropriate informative responses according to the dialog history. To achieve this goal, retrieve-augmented methods retrieve knowledge passages relevant to the original dialog history and concatenate retrieved passages and dialog history as the input of conversational agents. However, due to the excessive length and redundant content of retrieved knowledge passages, these methods decrease the computational efficiency and interfere with the input text understanding of conversational agents. To alleviate these two problems, we propose Keyword-Enhanced Learned Token Pruning (KELTP), which utilize the self-attention score and lexical knowledge to remove unimportant and irrelevant tokens accurately during inference. Experimental results on two knowledge-grounded dialog benchmarks confirm the effectiveness of our proposed method, which can bring 1.12 BLEU-1, 1.06 ROUGE-L, and 0.82 F1 score improvements with a 2.48x Speedup on the unseen test set of the Wizard of Wikipedia benchmark, unlike the previous work that has been done at the expense of performance for acceleration.