Multi-task Contrastive Learning Enhanced Instruction Tuning for Dialog Understanding
摘要
Dialog understanding (DU) of Task-Oriented Dialogue (TOD) aims to identify intents and their corresponding slots in user utterances. It is common for people to express multiple intentions in an utterance. Existing approaches for multi-intent DU face critical challenges in finding all intents and connecting slots to the correct intents, even for strong Large Language Model (LLM) based models. To address the challenges, this paper proposes a multi-task contrastive learning-enhanced parameter-efficient instruction tuning to make LLM achieve better multi-intent DU. For building contrastive samples, we first retrieve a positive sample and three negative samples for each utterance in training data. We construct six instructions for each of the five samples according to six different auxiliary task settings, including the original DU task, three procedural tasks and two dual-interactive tasks. Procedural tasks and dual-interactive tasks provide a step-by-step process of multi-intent DU as well as dependencies between intents and their corresponding slots, which is helpful in overcoming the above challenges. We group five instructions with the same task setting as a sample for LoRA instruction tuning. Experimental results on two publicly available multi-intent datasets, MixATIS and MixSNIPS, demonstrate that our approach outperforms current strong baseline models. It can improve the multi-intent DU efficiently without damaging the original capability of the LLMs.